Over a million developers have joined DZone.

Apache Metron Tech Preview 2 Released

The second release of Apache Metron is available with a ton of updates for working with cybersecurity and is a must evaluate for all companies.

· Big Data Zone

Hortonworks DataFlow is an integrated platform that makes data ingestion fast, easy, and secure. Download the white paper now.  Brought to you in partnership with Hortonworks

Two months ago, the Metron Engineering and PM team released Technical Preview 1 of Apache Metron based on the 0.1 release. We shared our vision for an open community based cybersecurity solution that provides real-time, cross-referenced and contextualized big data to combat cyber threats.

Apache Metron Reference Architecture

Metron Architecture - 12

As the above diagram illustrates, Apache Metron provides a real-time security stream processing pipeline to parse, enrich, apply threat intel, triage and store telemetry events generated from the diverse classes of data sources.

Metron exposes a Telemetry Ingest Buffer as a gateway into the pipeline which allows tools like Apache NiFi to stream data into the platform or with custom performant network data collectors that are part of Metron that stream data like pcap and netflow into the platform. Once the processing pipeline completes, Metron exposes a set of data services and integration that powers or will power a set of extensible modules that support the following capabilities:

  • Security Data Vault: Long-term storage of all telemetry data ingested, parsed, and enriched by Metron.
  • Search Portal: Index store that indexes the telemetry events and an UI portal to search for the events.
  • Provisioning, Management and Monitoring Tooling: Tooling to provision, manage and monitor the platform.
  • Community Analytical Models: A set of analytical models and packs developed in the community.

What’s New in Apache Metron TP2

Since the first tech preview released on April 19 2016, the Apache Metron community has been hard at work on Apache Metron Technical Preview 2 (TP2) which is based on the Apache Metron 0.2 release. Apache Metron is designed on four core functional themes to meet the specific user needs of SOC personnel and we are excited to announce today that Metron TP2 has new areas of functionality to address these personas. New capabilities available through TP2 are: Accelerated threat triage capability, that alleviates the time-consuming, serial nature of threat triage today, and expanded deployment options that allows Metron to be installed anywhere–on-prem, or in the cloud.

Metron TP2 Features and Enhancements

With TP2, we focused on three user personas: SOC Analyst, Investigator and the Security Platform Engineer. For these three personas, TP2 delivers the following capabilities across the four functional themes described above.  

TP2 Feaures

How to Get Started With TP2

With support of provisioning Metron on any Ambari managed HDP 2.4 cluster, you can now spin up Metron TP2 in two ways:

  • Ansible based Vagrant Single Node VM Install. This is a great place to start as an introduction to Apache Metron. Detailed installation instructions can be found here: Dev VM Install
  • Cloud-based install for a complete 10 Node Metron Cluster using Ambari Blueprints and AWS APIs. If you want a more realistic setup of the Metron app, you can install it on AWS. Keep in mind that this install will spin up 10 m4.xlarge EC2 instances by default. Detailed installation instructions can be found here: Cloud Install
  • Fully Automated Installation of  Metron on any HDP 2.4  cluster managed by Ambari. The cluster can be running on bare-metal, public/private cloud provider, etc. Detailed instructions can be found here: Metron Installation on an Ambari Managed Cluster

Where to Get Help

Hortonworks has created a new Community Cybersecurity Track in HCC.  Metron subject matter experts are answering questions and moderating the new Track for anything related to Apache Metron and Cybersecurity. When asking a question about Metron TP2, select “CyberSecurity” Track and add the following tags: “Metron” and “tech-preview”

Hortonworks Sandbox is a personal, portable Apache Hadoop® environment that comes with dozens of interactive Hadoop and it's ecosystem tutorials and the most exciting developments from the latest HDP distribution, brought to you in partnership with Hortonworks.

metron,big data,hdp,hortonworks,nifi,cybersecurity

Published at DZone with permission of George Vetticaden, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

The best of DZone straight to your inbox.

Please provide a valid email address.

Thanks for subscribing!

Awesome! Check your inbox to verify your email so you can start receiving the latest in tech news and resources.

{{ parent.title || parent.header.title}}

{{ parent.tldr }}

{{ parent.urlSource.name }}