SIEM Product Comparison – 101

SIEM Product Comparison – 101 

Please refer to the SIEM Comparison 2016 for the latest comparison.

We at Infosecnirvana.com have done several posts on SIEM. After the Dummies Guide on SIEM, we are following it up with a SIEM Product Comparison – 101 deck. So, here it is for your viewing pleasure. Let me know what you think by posting your comments below. The key products compared here are based on Gartner Magic Q which is what Organizations typically use to select SIEM vendors. The Vendors mentioned here in the deck are :

1. HP ArcSight

2. McAfee Nitro

3. IBM QRadar

4. Splunk SIEM

5. RSA Security Analytic

6. LogRhythm.

If you need any other Vendor evaluation on the parameters mentioned in the deck, please do let us know and we can post them for your use.

 

102 thoughts on “SIEM Product Comparison – 101”

  1. I would really like to see NetIQ Sentinel evaluated. I value your opinion and have actually use information from this site to educate management on this technology. My firm is wondering sho uld the choose between the products on the list or NetIQ Sentinel.

    1. Hi William, Sure. Here you go
      NetIQ Sentinel Strengths
      Sentinel and Sentinel Log Manager are appropriate for large-scale deployments that are focused on SEM and threat monitoring.
      The Change Guardian product line provides policy-based privileged User Activity Monitoring and Change Detection along with File Integrity Monitoring systems. This I think is because of their Novell roots, providing a clear option for Compliance related Monitoring and Reporting.
      Weakness
      Based on the Gartner Trends, you can see that Sentinel is not improving drastically in the last 3 years thereby bringing into question their focus on the technology and investment in R&D. Also Sentinel is a product bought over from its inception several times before finally landing with NetIQ. This kinda leaves them slower in the maturity cycle. Also, Sentinel lacks integration with threat intelligence feeds and their granular licensing practices can result in complex pricing evaluations and issues during renewal.
      Hopefully this helps

      1. Thank you. My background is with ArcSight so I am biased towards it. I found NetIQ some what lacking, they do not support NetFlow for example. Could you give me a numerical rating for this product? I know they are no way near ArcSight in maturity,

        Best regards,

    1. Hi Max,
      I am yet to evaluate Accelops, but from what I know, they are pretty new in the market and are focussed on “client customized” solutions rather than general capabilities. I know they have some promising features like Pattern discovery as against Correlation rules, but I am yet to take them to a spin. Will post a review as soon as I get my hands on an instance to play around

      1. Hi Misnomer, thanks. I don’t know what you mean ‘client customised’. My experience of it is that it’s a dashboard for both SIEM but also for very specific performance and availability/capacity monitoring too. All SIEM solutions have to be customised.

        1. Thanks Max for your feedback. As I said I have not played with it to make an informed judgement. What I meant by client customization is that I have spoken to outsourced vendors using Accelops and they mentioned that it’s new in the market and is vertical specific with a lot of client customization done. That’s all. If you have any insights on the product in detail, please do share. It will be lovely to know more about the product

    1. Haha…I think OSSIM can be included but they are not available with Enterprise Support you see… But the open source turned enterprise SIEM – AlientVault can be evaluated if you are interested.

  2. I’m ArcSight user by day (work) and OSSIM user by night (home network). I too would like to see where you would score OSSIM. To be fair AlienVault USM Enterprise would be the best choice for evaluation, yes because of the enterprise support. Thanks.

  3. Could we also take a look at the recent changes to the recently upgraded HexisCyber solutions Hawkeye G and AP?

    1. Hi Tim,

      I have no idea about the product and its capabilities. Please do share information if you have any about the products

  4. I’m corrently evaluting 5 leading Gartner’s SIEM tools, EMC-RSA and Symantec but I have not yet find out which tool is the best between this 7. the tool must be easy to deploy

  5. I have completed large deployments of Assuria, Splunk, Accelops and also been involved with deployments of Logrhythm, Arcsight, Solar Winds, I also recently re-evaluated the following products using strict criteria:
    -Logrhythm
    -Splunk
    -Tripwire
    -Accelops
    -RSA Envision
    -ArcSight
    -Mcafee
    -Assuria
    -Symantec
    -ARKAD
    Logrhythm, Tripwire and Accelops got into the shortlist.
    By far(!) the quickest to deploy was Accelops partly because its a virtual appliance with reporting and rules already packaged and it scales very well using additional virtual appliances. I am also formally trained by Splunk which is a good product but it is not a SIEM out of the box, even if you use the apps available it will still take too much time and money to get close to what some of the others offers as standard, i’m very surprised anyone considers Splunk a ‘SIEM’.

      1. Hi Rod, because SPLUNK is a log aggregator only, until you build the intelligence yourself. Its a great product, but it does not provide out of the box SIEM dashboard functionality that you would need in a SOC function. There is too much customisation required with SPLUNK to make it a SIEM, that’s my opinion from experience with the product and through my own evaluations against other products.

    1. Max, I have been tasked with deploying Splunk, I’m wondering how much trepidation will I go through trying to get this up and running, can you provide insight? I’ve also requested formal training from the customer in hopes that it will help me in Splunk overall. Mainly because I’ve been hearing just what you said, that Splunk is not fun out of the box. My request was Qradar for ease of deployment, but we couldn’t get the numbers together in time. What’s your assessment?

      1. If you are looking at Log Management, it should be pretty straightforward. The key things to consider are:
        1. Logs collected every day – This is going to give an assessment on how much capacity you would need. Keep in mind, that whatever Splunk may say, for best search results you should be using 100GB per server (indexer). So knowing your volume of logs will help you size the solution.
        2. Next decide on the log collection mechanism, install the relevant plugins for them and you should be able to parse most. Try to get the Splunk apps mostly, as the community parsers are a bit risky and may not be supported.
        3. Then deploy SIEM App. This is where it gets tricky. For basic compliance reporting, notifications etc, Splunk helps you out of the box, but the moment you want to implement custom use cases, you need to be fluent with the Splunk query language. It is not easy to learn at first, but if you get a hang of it, then it becomes intuitive. You can use several functions required for correlation using Field extractions, Mathematical functions, Tags etc. Splunk gives you the platform and the language set, but you are on your own to implement and generate value.
        4. Filtering – This is where I believe Splunk gives you limited options. Either you filter at the Source or at the Indexer. At the indexers, you don’t want to apply filtering as you want all the logs which leaves you to filter at source – A real pain. If you don’t focus on filtering, your splunk instance will start to crawl in a few months.

        QRadar would have been easier to implement, but I guess the choice has been made in your case.

        1. Thanks, this is most helpful! I’m sure I’ll have more questions as implementation and configurations kick off!

    2. I noticed you recently reviewed or evaluated RSA enVision around April 2014. Is there a reason you did not review RSA Security Analytics which is the path forward for RSA enVision?

      1. Actually, we have reviewed RSA Security Analytics. However, we have not been able to find the time to post a review. We will post it as soon as we can

  6. Good day Max and others

    Having so big experience with different SIEMs, what would you recommended for my case:

    1 Ability to detect anomalies and perform behavioral analysis in real-time.
    2 Support of NetFlow, JFlow, SFlow, Packeteer.
    3 Ability to analyze network traffic including Layer 7 (Application).
    4 Ability to correlate both event logs and network flows.
    5 Ability to monitor network flows in virtual environments.
    6 Ability to store both raw data and normalized data.
    7 Ability to instantly retrieve raw data associated with normalized data.
    8 Support for distributed architecture and horizontal scalability.
    9 Support for built-in high-availability mechanisms.
    10 Cost-effective, adequately fast and not complex system deployment.
    11 Single user-friendly interface for the whole SIEM system.
    12 Support for auto discovery of log sources.
    13 High-performance SIEM database.

    40000 EPS and 300 000 FPM

    The scale is large, about 15,000 hosts.

    Thank You

    1. Now the requirements you have posted call for a silver bullet SIEM solution. In my mind, IBM QRadar comes as the top choice to perform flow and event correlation however the event rates are something that will make the implementation very complex and difficult to manage.
      HP Arcsight satisfies high volume requirement and can perform Flow correlation too but it has a steep learning curve. Other than these two McAfee is the only other distant option

    2. If you want to just collect data then Splunk will do this and do it well. If you want to use the data and analyse it then Splunk will only do this with allot of expensive customisation (time and money). If you want to use the data from these device and get value out of if quickly then a product with out of the box reporting is what I recommend. As you are talking about quite a large implementation then just the discovery phase could take a while so you don’t want to then have to customise a dashboard and reporting too. Tripwire, or Accelops I think, Accelops is virtual appliance based with integrated DB so if your a VMware house then its a no brainer. It depends on timescales, budgets and what you want to use the data for….

      1. Very good point. We are actually having this discussion and I view Splunk as a Pre-SIEM product, but I am having a hard time describing this, yes SIM vs. SIEM is easy for SIEM users, but not so much for management, any ideas on maybe a good approach to take, I tried HP ArcSight Logger .vs Splunk

  7. Hi guys,

    I would definitely agree with Max. We do quite a lot of testing here too. We run a Lab where we usually deploy several security solutions to compare against standard and custom feature requirements. When it comes to security we do rank AccelOps very high. It is easy to deploy with intuitive search engine which runs both realtime and historically. You can use regex and structured (can be very specific and complex with inclusion and/or exclusion criteria, count, average, min, max and so on) searches. Many correlation features come out of the box but you may agree that SIEM also needs to be customizable in order to address specific infrastructure and security requirements. So you can easily add dashboards or customize correlation rules or add new ones from scratch or by simply copying and customizing existing.

    There is much more to be shared here so I would not take more time but yes I would definitely recommend Misnomer to test AccelOps and feed the results.

    @ Farik: AccelOps is very scalable and multi-tenancy is fully supported. Role-based access is very powerful feature too. As far as I know the biggest foundation comes for 10000 devices but you need to ask as the solution is meant for Security MSPs so it should definitely scale bigger. Stated EPS and flows counts are not a problem.

    1. Hi Lvelin,

      I would definitely like to review AccelOps. But I don’t have any information on the product. Can you guys provide some technical insight on how it works and I can compare against the products I know of.

  8. Thank you Lvelin and Mlsnomer.

    Does anybody has an experience of installing a SIEM on ISP level. I mean, it looks more complex than testing SIEM in lab. If anyone already done that before, or may be knows something, what would you reccomend, which pitfalls can I probably meet doing that task. Thank you in advance.

    1. Farik, if you mean ‘ISP level’ as in a service provider (managed service provider) with multiple customers (multi-tenancy) then Accelops does this natively, Tripwire offer similar functionality but its not as quick to deploy.

      1. It’s really interesting to see Tripwire as a recommendation. How has the experience been with Tripwire compared to other vendors?

        1. Tripwire is a good dashboard but it is software not an appliance and relies on Windows and SQL or MySQL which is not performant. You would need a huge amount of tuning to get Tripwire to work at the same speed as a Lucene style DB which Accelops uses. We tested Accelops with allot of data and used tier 2 sata storage and running reports over 30 day periods only takes seconds as opposed to hours or minutes on the Windows based Logrhythm, Assuria, Tripwire etc. Use a virtual appliance if want to scale, avoid software that sits on something like Windows that is not optimized for SIEM.
          Hope that helps.

        2. I am absolutely not recommending Tripwire. It just offers some similar multi-tenancy functionality, but as I say, it is flawed fundamentally as it is Windows and SQL based it simply would not perform with things like reports especially if you have multiple reports running simultaneously. Accelops and Splunk outperform it but Accelops deploys quickest and the reporting is easiest to use.

  9. Does anyone have any experience with NetIQ Sentinel? My firm is looking to use this SIEM.

      1. I do not, my background is with ArcSIght and IBM Q1Radar. The product is legacy because the firm is using NetIQ IAM products. Let us call it the path of least resistance in SIEM selection. I am looking for Threat Intelligence, support for NetFlow and able to integrate with ease with diverse event sources, not a lot of custom work. Which I get with more mature products. Just looking for anybody else that can really tell me their experience with it. I have a large enterprise and I think HP or IBM match the specs, not so keen on using Sentinel.

    1. Have you captured your requirements this helps create criteria on which products can be compared. Never even bother looking at products until you know and have agreed what your buying it for.

      1. Yes, I have the requirements and they fit HP or IBM, just trying to figure out if Sentinel can match these other products. I am concerned about the maturity of it.

        1. Honestly the product is good only for User Activity monitoring and basic Application monitoring. However for network devices and threat intel and stuff the support is poor. Also it involves a lot of custom work if your application landscape is big as the product does not have mature parsing support

        2. NetIQ Sentinel does include NetFlow support and Threat Intelligence feeds as of major release versions that were introduced at different points throughout this past year. It also includes a mapping feed (Advisor) that correlates your IDS/IPS (TippingPoint, etc) events to your Vulnerability Scanners (Nessus, etc) events so that you can know the impact of an observed attack combined with the vulnerability details of the target.

  10. Great overview deck!

    As previously mentioned in the thread how would Alienvault compare especially with its visionary positioning in the quadrant. How does its rule engine and correlation capabilities compare and does the enterprise wrap of open source bring anything more than support. Many thanks.

    1. AlienVault has a great foundation in the Open Source realm, however, when it comes to Correlation maturity, I think they still need improvement.
      Even from Parser support, other than AD, SNORT and other Open Source Applications (based), support is minimal.
      The GUI also is still not intuitive as compared to the SIEM vendors in the market and limits visualization, analytic and workflow

  11. Dear Max,

    Could you please say tell some words about installing SIEM on ISP(internet provider). Is it better to put it in-line, or pave through another route to avoid a latency? Thank you

    1. Hi Farik,

      What do you mean by Inline? In my opinion, SIEM should always be internal. Log collection can happen from Perimeter and Extended Perimeter using dedicated or shared Collector infrastructure. However, the Correlation Engine should be internal. As far as latency is concerned, since your logs are shipped internally, latency should be a non-issue.

  12. Has anyone tested any Tenable products such as Security Center, I’m trying to do just what this thread is all about, side by side comparisons. I read up on Q3 and was impressed, but on the fence between Splunk Q3 and Security Center. I’m looking the configurable that all speak of, scalability, ease of use, etc. Thanks!

    1. BDKang, there is no comparison between Security Centre and Splunk, Splunk needs to be customised and has very few out of the box rules. I recommend you compare, Accelops, Security Centre, Tripwire and McAfee. All are good but Accelops will be lowest price and scales the best.

      1. @Nothinghere2011..Thanks for the response. The company decided on Splunk..now I’m on the hook to try to make it work. I see a few threads down @William Kup. That he’s pro Splunk. I’m “ok” with it as well, if it wasn’t for the seemingly horror stories on here that it’s high maintenance and not easy to configure out of the box. Definitely NOT a comfortable feeling at all. Any recommendations on being able to tackle SPLUNK mountain?

  13. what about arctic wolf networks? would you be able to do a comparison on that? the CEO/founder also founded blue coat systems before he sold it… interesting!

  14. I would like to see AlienVault as they have really grown and I think they make a much more complete product for a price point that is great. I would love to get a second opinion on their product.

  15. I don’t understand the hit against Splunk in correlation. Since you can free form create any Splunk rules desired, it is easy to correlate on multiple fields and commands like “join” and “transaction” give you additional capabilities.

    1. William, Splunk Correlation capabilities when compared against the other SIEM vendors falls short. Splunk as you are aware comes from the Log Management background providing excellent search syntax and such, but SIEM is a plugin app which they released in the last couple years and is still maturing in terms of Correlation capabilities. That is why there is a hit against it

      1. Thanks for the reply. I wasn’t talking about Splunk’s Enterprise Security add-on but just Splunk Enterprise. I come from an ArcSight background but I find Splunk much more flexible in terms of creating arbitrary joins of events. Because I can do mathematical functions on the fields, I can put together arbitrary rules like “show me (event A followed by event B on same host within X minutes) where total bytes > 500 AND no corresponding domain login took place” I think they have some limitations, particularly where you need to create arbitrary length chains of events (event a -> event b ->…-> event n) but I haven’t found any tool that does that well.

        1. I’m curious about Splunk’s low scores vs. their continual improvement on the Gartner Magic Quadrant – what is Gartner seeing that you are not? This year Splunk is well past LogRhythm and challenging IBM and HP in the upper right quad. This is not a question with an agenda – just genuinely curious about the differences.

        2. Gartner Quad works on a weighted average based on market presence, customer feedback, product etc. The comparison in this blog is more on technical parameters only. Honestly Splunk has great search features and flexibility and is a very good log manager. But it has major caveats as well when it comes to SIEM for enterprise. One Example would be their 100GB per indexer limit. If I want a 500GB storage or 1TB storage I need 5-10 indexer instances to have a good Splunk performance where as IBM’s lowest model appliance can handle 2TB storage in 1 box. So cost, management, integration et all will cause overheads in Splunk but not the other SIEM vendors. Another example would be Correlation rules. Compared against IBM and HP, they are still behind in a lot of ways, even though they are constantly improving it.

          1. Where have you found there is a limit about 100 MB storage per indexer ?!?
            I have several TB per indexer, and it works really good !

            It seems people on that blog is like the comparison : not aware about SIEM market, for example how can we say ArcSight is the best….

          2. It’s 100GB per indexer for optimum search performance as per Splunk -http://blogs.splunk.com/2014/05/07/splunk-sizing-and-performance-doing-more-with-more/
            If you are unaware of building optimum architecture that applies to all general audience on Splunk platform, please refrain from making such comments. Performance and sizing is an art and there are several parameters to consider.

            No one said ArcSight is best. It’s a neutral comparison on various parameters. On some features ArcSight scores better and some features Other SIEM. We all know the market and that’s why this entire post has several good feedback points.

  16. Can you please provide feedback on Solarwinds. It is a tool already used within this organization and they are now considering purchasing the LEM module. What is the value versus bringing in a better known product such as QRadar. (I used this in the past and enjoyed the ease of use).

    1. In general, LEM provides basic correlation capabilities and reporting. It also comes cheap considering you are already a Solarwinds customer. However, keep in mind that it does not provide complex correlation use cases, does not have flow correlation like QRadar or ArcSight or McAfee does. Also, the support for product parsing is not as extensive as the usual suspects in SIEM.
      I think ease of use is similar to QRadar as Solarwinds comes as a VM, so it should be pretty straightforward. Also, if the requirements are basic monitoring and reporting, LEM should suffice.

    2. Hi, LEM is simply a log management tool, it is only value for money on that basis. It cannot be called a SIEM, it does not provide dashboard functionality such that you could use it as your single source to run an operations centre or SIEM SOC function. Its simply not function rich enough to-do that. As part of the comparison I mentioned above we were already using Solar Winds but replaced it with Accelops after comparing a great many products. I hope that helps.

    1. ya it was good.the recent release with 1000 reports and alerts are pretty useful.but we can’t say eventlog analyzer as compete siem.

    1. We use eIQNetworks SecureVue in my organization for almost a year now and seems to be a capable solution for my needs. However, would appreciate expert reviews on it

  17. Hi. My organisation procured Splunk to replace RSA Envision. Someone then decided to buy Change Guardian when they bought the NetIQ IAM suite. I need to write a position paper to justify why we don’t want to implement CG but instead use Splunk to collect and analyse events, as eel as provide reporting capability.
    How do I start? Which are the points that i should focus on? Does anyone think that we should use CG, although it is only for AD?
    Thanks for any response.

    1. If you want to take a position to have a single product to do everything, then Splunk should be the way to go. Personally, I would save CG licensing cost and use AD Monitoring app from Splunk to do what you seek to do with CG. Splunk Apps are a great way to improve your monitoring and they are very simple to use once you are already a Splunk customer. Refer to the link for AD monitoring on Splunk. http://docs.splunk.com/Documentation/Splunk/6.1.4/Data/AuditActiveDirectory

  18. I currently have Splunk and ArcSight deployed and have evaluated several other products. From a logging standpoint Splunk is by far the leader but I will have to agree with the earlier assessment that it is not a SIEM. Out of the box, ArcSight provides a large number of pre-written event based rules and reporting that will meet most of your audit requirements while everything is custom built in Splunk. From an after the fact investigative standpoint Splunk is a spectacular tool and I consider this the the “Google” of logging solutions. From an Operational security monitoring perspective ArcSight shines. From alerting to workflow and case management it can handle everything you could ever need. That being said, I do not believe any Enterprise level SIEM is truly ready out of the box and they all take attention and tuning to yield results when it comes to responding to security events.

    1. Completely agree with you on all the points. But Splunk can quickly get expensive when dealing with large amount of logs. Also, it becomes challenging to retrieve older data sets because of the way it stores and indexes logs. I have seen monthly reports on large Splunk installs take a hell a lot of time. I think none of the SIEM products do a good job in such larger time based queries.
      As far as the SIEM market goes, its absolutely true. Please read http://infosecnirvana.com/adopting-siem-what-you-need-to-know/ where we highlight exactly these points and more

  19. When you did the comparison with Splunk did you leverage the Splunk ES (Enterprise Security) App? I am wondering because that app leverages a CIM (Common Information Model) for all data flowing in to make the out of the box correlation rules very easy to run and trigger. I have set this up in less than a week from scratch using the documentation provided and the alerts/events generated were actually meaningful to the customer.

    1. YES, The SIEM app is considered. However, keep in mind that Splunk ES is not as advanced as other SIEM in the market.

  20. Can anyone point me to the information that supports Slunk is pre-SIEM please? I’m having a hard time articulating why this product does not equate to a ‘true’ SIEM solution. I appreciate the time in responding… Merry Christmas 🙂

  21. We’re thinking to opt for a cloud service. Please can anybody help me in identifying best SIEM cloud provider? In terms of service, features and cost.

  22. in individual product slide, McAfee Nitro weakness is mentioned as no analytic capability, while in the score card, the same product is rated highest with 4.5/5. Is there a confusion or something you would want to explain?

    1. Hi Gagan,
      Thanks for pointing out. The Analytic capabilities in the intro slide focus on Big Data and Risk only. However, the evaluation table is with SIEM Analytic capabilities only. Now the major difference is based on the fact that McAfee Nitro uses parsed log data for SIEM analytic, however when it comes to free range data sets like Splunk can handle, the capability is not there. Also, QRadar has something called a QRM, ArcSight has something called as TRM and all these components provide Risk Analytic capability which McAfee does not. This is the major difference. However, thanks for pointing this out.

    1. Hi Jensen,
      ELK is definitely in our road map for reviews. Keep watching this space by subscribing and you should see it as soon as possible

  23. Can anyone provide the cons and pros of LOGalyze an opensource SIEM tool? Anyonee experience is recommended? if for an organization which for about 10000 devices what will be best open source SIEM tool which can be recommended for implementation?

    Moreover like to know about the best opensource tool which can be used for encryption for data at rest and in transit?

    1. LogRhythm and QRadar are both simple and easier to deploy products. They have comparable application monitoring capabilities, however, when it comes to Network Forensics and Threat Intelligence, IBM outshines LogRhythm. In price, LogRhythm wins easily.
      So at the end of the day its your personal requirements, evaluation criteria and price that help you make the decision

        1. Accelops, Q and Logrythm are comparable enterprise SIEM products. In terms of reporting, alerting, rules, analytics. Logrythm have improved the collectors now supporting linux based collectors as VM’s. However Logrhythm will try to sell you hardware backend appliances, which don’t scale well but are fine for small-medium deployments and can get you started fast. Overall with Q you are buying into the brand. You won’t get more for your money by buying brands, Logrythm is better supported and better priced than Q. If you have to buy a strong brand then it’s between Logrythm and Q, Logrythm should win that one. If you want the comparable functionality as Logrythm but more flexible deployment model and cheaper licensing thenAccelops is better placed. It’s about knowing which functions you value and which you don’t. M

  24. Pingback: Splunk Start |
  25. We have seen demos on Black Stratus Log Storm, Accelops, Log Rhythm, and Solarwinds. We are now looking to do a demo of EventTracker – has anyone used it or done a POC? Looks their UI version 8 has been updated and is a big improvement over version 7.

  26. Hello misnomer: Any docs or links you could send my way to create SIEM in virtual environment?

    I did manual log correlation before (painful) and I am liking this idea with SIEM/SIM/SEM.

    1. All of the products listed in the SIEM comparison are available as virtual software. So I think you can pick any one depending on your requirements

  27. Is there a updated comparison available. This video seems to old and the products have evolved a Lot. Would appreciate if we can get a new version.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.