Community Articles

Find and share helpful community-sourced technical articles.
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
Super Collaborator

Metron TP1 Features

The following are key capabilities available in Metron TP1 broken up across its four key functional themes.


How do I get Started?

You can spin up the Metron TP1 in two ways:

  • Fully Automated 10 Node Ansible Based Install on AWS using Ambari Blueprints and AWS APIs
    • If you want a more realistic setup of the Metron app, use this approach. Keep in mind that this install will spin up 10 m4.xlarge EC2 instance by default
    • Detailed instructions how to do the install can be found in the following HCC Article: Apache Metron - First Steps in the Cloud

Where do I get Help?

Hortonworks has created new Track called CyberSecurity in the Hortonworks Community Connection (HCC). The link to the this new track in HCC is the following: HCC CyberSecurity Track.

Apache Metron committers are subscribed to this track and are constantly monitoring it for any questions the community has on TP1.

When asking a question about Metron TP1, please select the “CyberSecurity” Track and add the following tags: “Metron” and “tech-preview”.

Platform Theme Features of Metron TP1

The below is a summary of the key platform features added in TP1:

FeatureRelated Apache Metron JIRAS
Support for HDP 2.3
Refactor Metron Topologies for Performance, Easier Manageability & Supportability


Fully Automated Install of Metron on AWS on multi-node HDP cluster via Ansible scripts, Ambari blueprints and APIs.


Single Node Vagrant Support for Metron for Development


Unit and Integration Testing Frameworks, Code Test CoverageMETRON-82 METRON-58 METRON-37 METRON-28

Telemetry Data Source Theme Features of Metron TP1

Metron TP1 focus is network telemetry data sources as described below. They represent the most valuable granular data one can collect and perform next generation analytics on.

The Key Data collection features for Metron TP1 are the following:

FeatureRelated Apache Metron JIRAS
PCAP Ingest Data Services - Performant C++ probe that captures network packet and streams them into Kafka and gets bulk loaded into MetronMETRON-79 METRON-79 METRON-73 METRON-55 METRON-39
YAF/Netflow Ingest Data Services - Ingests netflow data into MetronMETRON-67 METRON-60
Bro Ingest Data Services - Custom BRO plugin that pushes out DPI (Deep Packet Inspection) metadata into MetronMETRON-25 METRON-73 METRON-64
Snort Ingest Data Services - Stream snort generated alerts via Flume into MetronMETRON-57
Grok Framework - Ability to add new Data Sources to Metron without writing new Parsing Topologies. For each new data source, grok expression file can be provided to normalized into Metron Event. METRON-66

Real-time Data Processing Theme Features of Metron TP1

For this theme, the key features in Metron TP1 are the following:

FeatureRelated Apache Metron JIRAS
Enrichment Services - OOO support for GeoIP and Host enrichments, extensible framework to plug-in new enrichments, & management Utilities for Enrichment Data METRON-32 METRON-43
Threat Intel Services - Integration with Soltra (Threat Intel Aggregrator) and Hail a Taxii, management Utilities for Threat Intel (Streaming and Bulk Load, aging out of data)


Alerting Services - Alerts can be fired via a snort event or intel threat feed hit

Indexing Services - Support for indexing via ElasticSearch


Storage Services - persisting all enrichment telemetry data in HDFS and or HBaseMETRON-62 METRON-22

UI Theme Features of Metron TP1

There was less focus on the UI Theme but Metron TP1 does provide the following new UI features:

FeatureRelated Apache Metron JIRAS
Metron Investigator IO Dashboard for the SOC Analyst and Investigator Personas built on top of KibanaMETRON-72 METRON-77 METRON-81
Histogram Panels for each of the data sources (YAF, Bro, Snort, PCAP)METRON-60 METRON-52
PCAP panel allow you to search for and download PCAP filesMETRON-72 METRON-77 METRON-81
Ability to customize the Metron UI with different data sources and different panel types.METRON-72 METRON-77 METRON-81

New Contributor

Is it possible to run this on Solr versus elastic? Solr is more closely tied to the Lucene project, uses the same Zookeeper based distributed coordination and continues to add more features to the open source project (graph query, parallel sql), while elastic seems to be taking more new features closed source. I've been building a lot of related features for network analysis in Solr.

Super Collaborator

Good question @Matt McKnight. We will have support for Solr indexing services in Metron TP2 which is slated for end of May. However in TP2, we will still only support Metron UI that is based on Kibana (based on Elastic). This will change in subsequent reelases. So net net, by middle/end of May we will support Solr indexing but you would have to write the UI that calls the SOLR Apis for search queries. Farther down the line, we will provide a custom UI (away from Kibana) that uses SOLR to do search.

Make sense?