At first blush, logging seems simple. Turn it on, collect what you log, review and analyze what you collect. For individual systems and small-business LANs, logging can be as simple and easy as drinking from a garden hose. Now imagine drinking from a fire hose (or several) and appreciate how quickly you can go from drinking to drowning in a sea of seemingly unrelated data of arbitrary format, context, and content.
One way to help your partners get the maximum benefit from logging activities is to explain the value of having a game plan for collection that is synergistic with log monitoring and analysis needs. Here are some elements to consider.
Choose relevant data from critical systems
As you develop your game plan, consider what applications, systems, and network segments you should log and collect data from. Logging not only helps you distinguish new or suspicious activities from expected activities, but it can help you understand whether your resources are being optimally used. It’s important to collect data sufficient to establish a baseline for what is normal activity, expected load.
Consider, too, how you intend to use the data and the level of detail you’ll need to respond or report. For example, when planning for real-time monitoring, consider what events or thresholds should trigger alarms. For performance, consider what information you’d need to anticipate a change in application mix.
A good initial reference point for this part of your game planning is Purdue University’s Basic Logging Standard, which uses such factors as criticality of resource or underlying data; past experience of a resources vulnerability to attack or misuse; and extent of system interconnectedness to determine what to log and to what level of detail.
Respect your resources
A directive like "log as much as your pocketbook can spare" offers guidance for storage considerations. Log or event recording consumes processing cycles and memory, so it’s important to be mindful of system and network performance, too.
Make your log data readable, consistent, and easy to correlate
Collecting event or log data from various sources typically means that the format and content of each record may be different. Leverage common log formats (e.g., NCSA for Web servers) and normalize non-standard log records to conform to the standard you choose to simplify analysis. Synchronize time across your network, timestamp every event or record, and, where you have the ability to customize, create events that humans can read. Splunklabs offers some useful advice for managing this facet of your game plan.
Care for and maintain your log data
Explain to your partners that how they store (archive) and preserve the integrity of their log data is an important consideration for any logging game plan. A retention game plan should reflect where and how you will archive logs, how long you believe log data will remain useful, and how historical data is accessed.
Retention policies ought to consider prevention of physical loss, damage, or destruction, as well as data integrity loss (corruption or alteration). How frequently you may need to access historical data will affect where and how you retain log data. Your entire retention plan may be strongly influenced by regulatory obligations (e.g., healthcare).
The elements we’ve discussed here offer a garden-hose feed of information to consider when formulating your logging game plan. If your partners beg for more, point them to other readily available, quality resources such as NIST, OWASP, SANS, and Dr. Anton Chuvakin.
Originally posted at The Champion Community 1 October 2012
Comments