Adopting SIEM – What you need to know?

SIEM stands for Security Information and Event Management”

Oh wait, I have heard of SIM, I have heard of SEM, what is this SIEM??.
Originally, the Security Information Management (SIM) and Security Event Management (SEM) systems were two different technologies performing similar but distinct functions. Gartner in 2005, coined the term SIEM to encompass both. As the name suggests, It is nothing but a collection of tools and technologies to manage Incident and Events pertaining to Security alone. Some of the tell-tale capabilities of a typical SIEM platform are:

  1. Collect Logs from various Log Sources/Devices
  2. Store these logs for a decent amount of time
  3. Provide Fast Search/Retrieval capabilities
  4. Provide meaningful interpretation of Log received
  5. Provide capabilities to correlate between logs of different devices
  6. Basic Ticketing/Alerting capabilities.

The first 4 points are typical of a SIM and the remaining 2 are typical of a SEM.
Any tool that does all of these is a SIEM. There are more than 50 different products that cater to the SIEM space. Just like any other product, they cater to various market segments at various price points.
If you Google for SIEM reviews you would get a lot of information on various products. In my experience, I have worked with at least 4 SIEM vendors. Each one of them have their own pros and cons. Comparing a product in a DEMO and comparing it after use are two different things. So, in this blog post, I am going to highlight few things as “What you need to know” when you are planning to adopt SIEM technology

  1. Have a defined Logging process in your environment. This is very crucial because a SIEM is useless without a good Logging Program. This not only helps in making the SIEM implementation easier, but also helps in getting a measure of the volume you are dealing with. In my experience, often times, despite having an Industry leading SIEM, the log Management made it look pedestrian and a waste of money.
  2. Every SIEM vendor has something called as Collector/Connector/Receiver/Agent that collects logs from the devices and converts them to their proprietary format. This conversion or parsing as we call is important for the product developers to store data in a format they can understand and process quickly. Most of the vendors offer something of a Custom Collector/Parser development for their “unsupported” log sources. This costs money, skills in-house and may require regular maintenance. Hence Native Parsing Support for Log Sources is better. Establish this before you move ahead with SIEM implementations. Either source a in-house resource to help build and manage such customizations or spend more money to get the vendor to do it.
  3. Identify primary focus areas from an Organizational perspective. This will help you configure your SIEM solutions appropriately. These focus areas should be broadly classified and then expanded to the ground level. For example, if your requirement is compliance, start with control requirements, see what logs need to be collected to fulfill them, see how integration needs to be done, see what needs to be reported, alerted, retained, etc.
  4. Get a dedicated SIEM administrator or rather train someone in-house to be that person. This is very important because, in my experience I have always felt that SIEM is as good as the administrator is. Without proper maintenance and care, it will decay over time. If you really need to generate value out of it, manage it well. By managing a SIEM I mean not only the system itself but also the ecosystem it resides in.
  5. Understand that SIEM alone cannot solve all your Security Problems. It is NOT A MAGIC WAND. If setup and configured correctly, a SIEM can at best point you in the right direction, a direction where you can identify and fix several security issues in your enterprise thereby strengthening it. So, be prepared to have a Response/Remediation team that will investigate the alerts generated and take appropriate action.
  6. Correlation is a vital part of SIEM offerings. Before Adopting SIEM, make sure you understand and possibly catalog the various Attack Vectors, Threat Scenarios you would want looked at for correlation in your organization. This will give a fair direction for the basic rules you would put in place to start with. Once you are comfortable and start seeing the various alerts generated, you can play around and experiment more. In my experience, start with built-in rules, understand them, investigate them, tune them and then slowly start building your own content. For more details on the various rules available in SIEM Look at Rules Rule in SIEM Kingdom
  7. Architecture wise, make sure your SIEM solutions are in tandem with your Logging solutions. Also, build your SIEM as modular as possible thereby making upgrades, technology refresh etc seamless.
  8. Don’t forget the filtering aspect. Correlation Engines will perform faster and will get you better results if they are attacking a smaller set of “known bad” logs rather than all. This is crucial in large enterprises as the Log Volume can easily overwhelm the SIEM systems. Note: Many SIEM tools have limitations in the number of events they can process. This is denoted in Events Per Second (EPS). Even though the vendors advertise several thousands, an effective correlation system can have only around 2000 – 5000 EPS tops. Anything more will make your system painstakingly slow. So understand and work through this. Look at my posts What and How much to Collect and High Log Volume – What to Filter and What to Keep? to get more information on how to log, what to log and what to filter.
  9. Remember, more processing layers, less EPS. This means that the Log Collection layer will have more EPS processing capability than the Correlation engine and so on. Visualize it as a pyramid with the Log Collection at the Base and the Correlation at the top
  10. Last but not the least, “Stay Alert and Eager. The Logs Don’t Lie”

Hope this post helped you in getting a fair idea of SIEM technologies. I have worked on HP ArcSight, Symantec SSIM, Novell E-Sentinel. If you need details about them in terms of practical setup, configuration, architecture etc, shout out and I will help as much as possible.
 Save as PDF

13 thoughts on “Adopting SIEM – What you need to know?”

  1. Great information, I am planning to have such product for my company and doing POC of archsight and RSAEnvision.
    Thanks once again.
    Sunil Modhave
    Manager – Information Security

  2. This info is right on, and I haven’t seen such a completely cogent guideline in a long time. I particularly like the commentary about demos rarely live up to final implementations, and how EPS values are usually inflated. Very true.

    Finally, I really appreciate the embedded links. Great work, thanks!

  3. Great intro article – I urgently need verified research material on top 3 SIEM solutions/products with proper analysis or comparison done between them to allow me to make correct decision for my company as I am the security architect that will be responsible for making this recommendation.

    Can you please assist in this regard or provide me with the reference info or sights to obtain these solution comparisons urgently please??

  4. Decent write-up.

    Any organization looking into SIEM deployment needs to make sure to align themselves with a proven expert team. An SME would be able to guide the customer through the decisions on architecture, deployment, and the ongoing management of the solution, and will help avoid common pitfalls.


    1. Thanks Anton for the comments and You are right about mentioning that an SME is needed to get the ball rolling.

  5. Nice job. This is a good starter list, and I agree with each of your major points.

    You might also consider the key objectives.

    Accvuant recommendations and industry best practices around SIEM and Log Management focus on two keys:
    1. 99.99% Correlation Effectiveness – a reduction of events (noise), to offenses (threats) of 100,000 to one.
    2. Actionable events – each correlated event from a SIEM should be enriched with enough context (through network modeling, asset vulnerability data, black and white lists, etc…), to be actionable without hours of research in other tools.

    In order to achieve these results, our focus is to help customers identify and bring in logs from key data sources necessary to provide a complete picture of threats and countermeasures in a “single pane of glass”, and tune content to “find the needles in the haystack.”

    Additionally, Accuvant’s SIEM practice focuses on delivering reports that meet compliance requirements, and use log analysis to provide oversight of security sources and to enhance defenses in a continuous feedback loop, as well as providing visibility to management of current threats and the effectiveness of deployed defenses to help target responses.

    Public SIEM Practice Documents

    A Process for Continuous Security Improvement Using Log Analysis

    Successful SIEM and Log Management Strategies for Audit and Compliance
    A Compliance Primer for IT Professionals

    A Practical Application for SIM/SEM/SIEM Automating Threat Identification

Leave a Reply to NetIQ Cancel reply