Knowing how much people love checklists, here is one more: a checklist for comparing log management tools.
It is being released at the new log management related site, Log Management Central (subscribe to RSS, follow on Twitter).
- The announcement and brief description is here.
- Printable PDF version is here.
- Spreadsheet XLS version with adjustable criteria scoring is here.
Disclosure: creation of this checklist was funded by a vendor, but it did not affect my choice of criteria or any other content decision. It also does not reduce awesomeness in any way! In other words, it is up to you how to use it (and whether to use it) and what decision to make after evaluating the tools. Just don’t make a decision of letting your logs rot
Please feel free to make suggestions to make the checklist more useful! Is anything missing? Worded in a non-vendor neutral way? Anything else?
Possibly related posts:
As promised, here is another “Top 11 Reasons” which is about log analysis. Don’t just read your logs; analyze them. Why? Here are the reasons:
- Seen an obscure log message lately? Me too – in fact, everybody have. How do you know what it means (and logs usually do mean something) without analysis? At the very least, you need to bring additional context to know what some logs mean.
- Logs often measure in gigabytes and soon will in terabytes; log volume grows all the time – it passed a limit of what a human can read a long time ago, it then made simple filtering ‘what logs to read’ impossible as well: automated log analysis is the only choice.
- Do you peruse your logs in real time? This is simply absurd! However, automated real-time analysis is entirely possible (and some logs do crave for your attention ASAP!)
- Can you read multiple logs at the same time? Yes, kind of, if you print them out on multiple pages to correlate (yes, I’ve seen this done ). Is this efficient? God, no! Correlation across logs of different types is one of the most useful approaches to log analysis.
- A lot of insight hides in “sparse” logs, logs where one record rarely matters, but a large aggregate does (e.g. from one “connection allowed” firewall log to a scan pattern). Thus, the only way to extract that insight from a pool of data is through algorithms (or, as some say, visualization)
- Ever did a manual log baselining? This is where you read the logs and learn which ones are normal for your environment. Wonna do it again? Log baseline learning is a useful and simple log analysis technique, but humans can only do it for so much.
- OK, let’s pick the important logs to review. Which one are those? The right answer is “we don’t know, until we see them.” Thus, to even figure out which logs to read, you need automated analysis.
- Log analysis for compliance? Why, yes! Compliance is NOT only about log storage (e.g. see PCI DSS). How to highlight compliance-relevant messages? How to see which messages will lead to a violation? How do you satisfy those “daily log review” requirements? Thru automated analysis, of course!
- Logs allow you to profile your users, your data and your resources/assets. Really? Yes, really: such profiling can then tell you if those users behave in an unusual manner (in fact, the oldest log analysis systems worked like that). Such techniques may help reach the holy grail of log analysis: automatically tell you what matters for you!
- Ever tried to hire a log analysis expert? Those are few and far between. What if your junior analysts can suddenly analyze logs just as well? One log analysis system creator told me that his log data mining system enabled exactly that. Thus, saving a lot of money to his organization.
- Finally, can you predict future with your logs? I hope so! Research on predictive analytics is ongoing, but you can only do it with automated analysis tools, not with just your head alone (no matter how big ) …
Following my now-famous Top 11 Reasons to Collect and Preserve Computer Logs and Top 11 Reasons to Look at Your Logs, here is the promised “Top 11 Reasons to Secure and Protect Your Logs”
- Let’s review why you are reviewing logs. Will logs that might have been changed by somebody, somewhere, somehow still be useful for items 1-11 from here? No? Secure them!
- Oooh, logs in court? Challenges abound! To respond to them, one needs to protect the logs so you can claim that they are both authentic and reliable.
- A human error still beats an evil hacker as the main cause of IT problems. Are your logs safe from it? Available when needed? Protect them from crashes and other faults!
- PCI DSS just says so: “Secure audit trails so they cannot be altered.” Wonna do it- or pay the fines?
- Do you protect financial records? Identity info? Passwords? Some of it ends up in logs – thus making them more sensitive. Secure the C-I-A of logs!
- Do you look at logs during incident investigation? Do you want them to be “true” or full of random (if creative…) cr*p, inserted by the guilty party? Secure the logs!
- Think that “attacks vs logging” are theoretical? Think again. Are your logs safe or vulnerable? Is your logging tool 0wned?
- Syslog + UDP = log injection. Are you protected (reliable TCP, confirmed delivery, encryption – SSH, SSL, VPN)?
- Why change logs? No, really, why change logs? If you never change logs – and you never should – hash them right away after collection to make them immutable.
- Logs are backed up on tape – who will see them? Well, whoever restores the tape, that’s who! Encrypt them to protect them from accidental and malicious disclosure if tape is lost.
- Why log access to logs? Same reason why you had the logs in the first place – to review who did what. Who broke through and stole the logs? Who browsed them without permission? Only logs will tell – if you have them!
Overall, one need to strive for having no holes in log safeguards from log birth to analyst conclusion based on log information…
As promised, I am following my Top 11 Reasons to Collect and Preserve Computer Logs with just as humorous and hopefully no less insightful ”Top 11 Reasons to Look at Your Logs.”
- The first reason is again disarmingly simple (is it ). Read PCI DSS lately? Glanced at HIPAA? Suffer underFISMA? Yup, all of the above say that you must not only have, but also review logs periodically.
- Are you 0wned? How do you know if all your logs are stashed on a tape in a closet? Look at them! Now!!
- An incident happens. Really, who needs extra motivation to look at logs in this case? Duh! Logs for incident response is a “no-brainer” use case for log review.
- Users – from CEO to a janitor. You might have to know what they do on your IT systems! How? Read the logs! Everybody leaves tracks.
- Logged system errors. Sometimes they are stupid, sometimes – benign. However, often they mean that “stuff” is about to hit the fan. Periodic review of logs reveals them and saves the day.
- Network slowed to a crawl? Applications are slooow? Server is not … well, serving? Where is the answer? In the logs, but you need to read them and understand them.
- That policy you wrote a few months ago. Anybody following that? Anybody remembers that? Halloooo! Check the logs and you’d know.
- You know your auditor might check your logs. But did you know they might also check whether you looked at them? Did’ya? Review the logs and leave the record of this activity!
- Change can be good. But then again, it may be the sign that your controls are lacking. Who changes what and when? From what and to what? Just review the logs.
- Now, you hate looking at logs. You have too many! In this case, look at a specific subset of logs that you never saw before- NBS. Or just deploy log management that can do it for you.
- Logs can help you predict the future (if you review, know and love them ). Don’t believe it? If you read them for long enough, you develop an ability to – gasp!- predict the future, albeit mostly future problems
I’ve been wanting to create those for a loooooong time and finally – here they are (you can guess I’ve been on a long flight ). Some are admittedly tongue-in-cheek, but useful nonetheless. So, enjoy Anton’s ”Top 11 Reasons to Collect and Preserve Computer Logs”, presented in no particular order:
- Before anything else, do you deal with credit cards? Patient info? Are you a government org under FISMA? A financial org? You have to keep’em – stop reading further.
- What if there is a law or a regulation that requires you to retain logs – and you don’t know about it yet? Does the world “compliance” ring a bell?
- An auditor comes and asks for logs. Do you want to respond “Eh, what do you mean?”?
- A system starts crashing and keeps doing so. Where is the answer? Oops, it was in the logs – you just didn’t retain them …
- Somebody posts a piece of your future quarterly report online. Did John Smith did it? How? If not him, who did? Let’s see who touched this document, got logs?
- A malware is rampant on your network. Where it came from? Who spreads it? Just check the logs - but only if you have them saved.
- Your boss comes and says ‘I emailed you this and you ignored it!!’ – ‘No, you didn’t!!!’ Who is right? Only email logs can tell!
- Network is slow; somebody is hogging the bandwidth. Let’s catch the bastard! Is your firewall logging? Keep the info at least until you can investigate.
- Somebody added a table to your database. Maybe he did something else too – no change control forms were filed. Got database log management? How else would you know?
- Disk space is cheap; tape is cheaper still. Save a log! Got SAN or NAS? Save a few of them!
- If you plan to throw away a log record, think – are you 100% sure you won’t need it, ever? Exactly! Keep it.
Have more? Feel free to suggest your own reasons below!