Helping to protect IT environments from cyber attacks and comply with tightening compliance standards, SIEM systems are becoming the cornerstone for security paradigms implemented by a growing number of organizations.

There are proprietary platforms that do offer an all-in-one SIEM solution, such as LogRhythm, QRadar, and ArcSight. These solutions can become rather expensive, especially in the long run and in larger organizations, and so more and more companies are on the search for an open source SIEM platform.

But is there an open-source platform that includes all the basic SIEM ingredients?

The simple answer is: no. There is no all-in-one perfect open-source SIEM system. Existing solutions either lack core SIEM capabilities, such as event correlation and reporting or require combining with other tools. As always, though, there are some good contenders, and in this article, we take a look at six of these platforms.

We will follow up this article with a similar analysis of proprietary tools.

1. OSSIM

The open source version of AlienVault’s Unified Security Management (USM) offering, OSSIM is probably one of the more popular open-source SIEM platforms. OSSIM includes key SIEM components, namely event collection, processing and normalization, and most importantly — event correlation.

OSSIM combines native log storage and correlation capabilities with numerous open source projects in order to build a complete SIEM. The list of open source projects included in OSSIM includes FProbe, Munin, Nagios, NFSen/NFDump, OpenVAS, OSSEC, PRADS, Snort, Suricata, and TCPTrack.

The inclusion of OpenVAS is of particular interest, as OpenVAS is used both for vulnerability assessment by correlating IDS logs with vulnerability scanner results.

As one would expect, the open-source OSSIM is not as feature-rich as its commercial “older brother.” Both solutions work fine for small deployments, but OSSIM users experience significant performance issues at scale, ultimately driving them towards the commercial offering. Log management capabilities in the open source version of OSSIM, for example, are virtually non-existent.

2. The ELK Stack

The ELK stack, or the Elastic Stack, as it is being renamed these days, is arguably the most popular open-source tool used today as a building block in a SIEM system. A building block, yes. A complete SIEM system, no, since there is plenty of room for debate about whether or not the ELK Stack qualifies as an "all in one" SIEM system.

The ELK stack consists of the open-source products Elasticsearch, Logstash, Kibana and the Beats family of log shippers.

Logstash is a log aggregator that can collect and process data from almost any data source. It can filter, process, correlate, and generally enhance any log data that it collects. Elasticsearch is the storage engine and one of the best solutions in its field for storing and indexing time-series data. Kibana is the visualization layer in the stack and an extremely powerful one at that. Beats include a variety of light-weight log shippers that are responsible for collecting the data and shipping it into the stack via Logstash.

Logstash uses a wide array of input plugins to collect logs. However, it can also accept input from more purpose-built solutions like OSSEC or Snort (see below). Combined, the ELK Stack's log processing, storage, and visualization capabilities are functionally unmatched. For the purposes of SIEM however, the ELK Stack — at least in its raw open source format, is missing some key components.

First and foremost, there is no built-in reporting or alerting capability. This is a known pain point not only for users trying to use the stack for security but also for more common use cases — IT operations for example. Alerting can be added by using the X-Pack, a commercial product by Elastic, or by adding an adding open source security add-ons.

There are also no built-in security rules that can be used. This makes the stack a bit more costly to handle, both in terms of resources and operational costs.

3. OSSEC

OSSEC is a popular open-source Host Intrusion Detection System (HIDS) that works with various operating systems, including Linux, Windows, MacOS, Solaris, as well as OpenBSD and FreeBSD.

OSSEC itself is broken into two main components: the manager (or server), responsible for collecting the log data from the different data sources, and the agents — applications that are responsible for collecting and processing the logs and making them easier to analyze.

OSSEC directly monitors a number of parameters on a host. This includes log files, file integrity, rootkit detection, and Windows registry monitoring. OSSEC can perform log analysis from other network services, including most of the popular open-source FTP, mail, DNS, database, web, firewall, and network-based IDS solutions. OSSEC can also analyze logs from a number of commercial network services and security solutions.

OSSEC has a number of alerting options and can be used as part of automated intrusion detection or active response solutions. OSSEC has a primitive log storage engine. By default, log messages from host agents are not retained. Once analyzed, OSSEC deletes these logs unless the <logall> option is included in the OSSEC manager's file. If this option is enabled, OSSEC stores the incoming logs from agents in a text file that is rotated daily.

4. Apache Metron

From an architectural perspective, Metron relies on other Apache projects for collecting, streaming and processing security data. Apache Nifi and Metron probes collect data from security data sources which is then pushed into separate Apache Kafka topics. Events are subsequently parsed and normalized into standard JSON and then enriched and in some cases labeled. Alerts can be triggered if certain event types are identified. For visualization, Kibana is used (albeit an outdated version)

For storage, events are indexed and persisted in Apache Hadoop and either Elasticsearch or Solr based on the organization's preferences. On top of this data, Metron provides an interface for centralizing the analysis of the data with alert summaries and enriched data.

Being relatively young, Metron still lacks in some aspects. Metron can only be installed on a limited number of operating systems and environments though it does support automation scenarios with Ansible and installation via Docker (Mac and Windows only). The UI is a bit immature and does not support authentication for example.

5. SIEMonster

SIEMonster is another young SIEM player but an extremely popular one as well, with over 100,000 downloads in just two years. SIEMonster is based on open source technology and is available for free and as a paid solution (Premium and MSSP multi-tenancy).

While SIEMonster uses its own "monster" terminology to name the different SIEM functions within the system (e.g. Kraken), the underlying components are well known open source technologies. The ELK Stack is used for collection (Filebeat and Logstash), processing, storage and visualization of the security data collected. RabbitMQ is used for queuing. SearchGuard is used for encryption and authentication on top of Elasticsearch and ElastAlert for alerting. A fork of OSSEC Wazuh for HIDS. The list goes on.

From a functionality perspective, SIEMonster includes all the goodies an analyst could wish for, each accessed via the main menu — the Kibana UI for searching and visualizing data, a UI for threat intelligence, Alerts for creating and managing event-based notifications. Additional integrated open source tools are DRADIS, OpenAudit, and FIR.

SIEMonster can be deployed on the cloud using Docker containers, meaning easier portability across systems, but also on VMs and bare metal (Mac, Ubuntu, CentOS, and Debian). Documentation is extensive, though an online version is missing.

Similar to OSSIM, is a SIEM framework that unifies various other open source tools. And like OSSIM, it is also an open source version of the commercial tool by the same name. Prelude aims to fill the roles that tools like OSSEC and Snort leave out.

6. Prelude

Similar to OSSIM, Prelude is a SIEM framework that unifies various other open source tools. And like OSSIM, it is also an open source version of the commercial tool by the same name. Prelude aims to fill the roles that tools like OSSEC and Snort leave out.

Prelude accepts logs and events from multiple sources and stores them all in a single location using the Intrusion Detection Message Exchange Format (IDMEF). It provides filtering, correlation, alerting, analysis, and visualization capabilities.

Again, like OSSIM, the open source version of Prelude is significantly limited when compared to the commercial offering in all of these capabilities which is probably why it is not very popular. Quoting the official documentation: “Prelude OSS is aimed for evaluation, research and test purpose on very small environments. Please note that Prelude OSS performances are way lower than the Prelude SIEM edition.”

No "One Ring to Rule Them All"

A complete SIEM solution includes the ability to collect information from various data sources, retain that information for an extended period of time, correlate between different events, create correlation rules or alerts, analyze the data and monitor it with visualizations and dashboards.

Answering a lot of these requirements, it is no coincidence that the ELK Stack is used by many of the open source SIEM systems listed in this article. OSSEC Wazuh, SIEMonster, Metron - all have ELK beneath the hood. But taken on its own, ELK lacks some key SIEM components, such as correlation rules and incident management.

Based on the analysis above, the simple conclusion is that there are no clear winners to the title "an all-in-one open source SIEM solution." When implementing a SIEM system based on the solutions above, you will most likely find yourself limited as far as functionality is concerned or combining with additional open source tools.

Open source tools used for SIEM are versatile and powerful. But, they require a great deal of expertise, and above all — time to deploy properly. It is for this reason that commercial offerings still dominate the SIEM landscape, even when open-source tools lie at the core of those commercial offerings.

Having 80% of your SIEM solution handled for you is better than having to do it all by yourself. Commercial solutions handle installation, basic configuration, and provide filters, correlation configurations, and visualization designs for the most common use cases. Don't underestimate the value of these commercial features: there are a seemingly unlimited number of things to monitor in today's datacenters, and none of us have time to manually configure applications to watch them all.