Two months ago, the Metron Engineering and PM team released Technical Preview 1 of Apache Metron based on the 0.1 release. We shared our vision for an open community based cybersecurity solution that provides real-time, cross-referenced and contextualized big data to combat cyber threats.
Apache Metron Reference Architecture
As the above diagram illustrates, Apache Metron provides a real-time security stream processing pipeline to parse, enrich, apply threat intel, triage and store telemetry events generated from the diverse classes of data sources.
Metron exposes a Telemetry Ingest Buffer as a gateway into the pipeline which allows tools like Apache NiFi to stream data into the platform or with custom performant network data collectors that are part of Metron that stream data like pcap and netflow into the platform. Once the processing pipeline completes, Metron exposes a set of data services and integration that powers or will power a set of extensible modules that support the following capabilities:
- Security Data Vault: Long-term storage of all telemetry data ingested, parsed, and enriched by Metron.
- Search Portal: Index store that indexes the telemetry events and an UI portal to search for the events.
- Provisioning, Management and Monitoring Tooling: Tooling to provision, manage and monitor the platform.
- Community Analytical Models: A set of analytical models and packs developed in the community.
What’s New in Apache Metron TP2
Since the first tech preview released on April 19 2016, the Apache Metron community has been hard at work on Apache Metron Technical Preview 2 (TP2) which is based on the Apache Metron 0.2 release. Apache Metron is designed on four core functional themes to meet the specific user needs of SOC personnel and we are excited to announce today that Metron TP2 has new areas of functionality to address these personas. New capabilities available through TP2 are: Accelerated threat triage capability, that alleviates the time-consuming, serial nature of threat triage today, and expanded deployment options that allows Metron to be installed anywhere–on-prem, or in the cloud.
Metron TP2 Features and Enhancements
With TP2, we focused on three user personas: SOC Analyst, Investigator and the Security Platform Engineer. For these three personas, TP2 delivers the following capabilities across the four functional themes described above.
How to Get Started With TP2
With support of provisioning Metron on any Ambari managed HDP 2.4 cluster, you can now spin up Metron TP2 in two ways:
- Ansible based Vagrant Single Node VM Install. This is a great place to start as an introduction to Apache Metron. Detailed installation instructions can be found here: Dev VM Install
- Cloud-based install for a complete 10 Node Metron Cluster using Ambari Blueprints and AWS APIs. If you want a more realistic setup of the Metron app, you can install it on AWS. Keep in mind that this install will spin up 10 m4.xlarge EC2 instances by default. Detailed installation instructions can be found here: Cloud Install
- Fully Automated Installation of Metron on any HDP 2.4 cluster managed by Ambari. The cluster can be running on bare-metal, public/private cloud provider, etc. Detailed instructions can be found here: Metron Installation on an Ambari Managed Cluster
Where to Get Help
Hortonworks has created a new Community Cybersecurity Track in HCC. Metron subject matter experts are answering questions and moderating the new Track for anything related to Apache Metron and Cybersecurity. When asking a question about Metron TP2, select “CyberSecurity” Track and add the following tags: “Metron” and “tech-preview”