Background
Have you seen the online SIEM calculators used by some vendors? These calculators appear to give a view of log collection that indicates great visibility utilising little storage, however did you read the terms and conditions? If you had done so, you would realise that they are nothing more than a marketing gimmick.
Why do vendors choose to use them? The answer to this is that organisations need to know how much storage their logs consume in order to put budget against a SIEM solution, in fact some SIEM vendor charge by the storage amount consumed so it has a direct correlation.
This is why we developed our own tool to precisely calculate the storage used, I would like to introduce you to Log Analyser.
Saving Business Budget
DefenceLog uses a technique to save space by not retaining unnecessary parts of the log entry. Our tool calculates this saving. On average the space saving is between 25-30% depending on configuration!!
The “ParsedBytes” field is the size of the log after savings have been applied, whereas the “RawBytes” is the original size, this is done for all logs.
Recently, Defence Logic demonstrated for a customer that detailed Windows and Sysmon event logging (120 servers), would only take 19GB storage per day or 570GB per month!
How it works
Log Analyser is a multi-thread GUI application which acts as a proxy. Logs can be from multiple hosts and the size of the log in bytes is calculated and appended to the log information. This information is then recorded in csv format and or sent to our SIEM solution.
The recorded information (both Windows and Sysmon events) in the csv files can be used with a Juypter notebook or a MS Excel spreadsheet to provide an accurate report to the client including nice graphs, that way we are able to accurately predict storage.
It also has an added benefit of being able to be used internally when developing logging configurations.
Future updates will most likely include automatic report generation.
You can see it in action, in the short video below.