Metron Statistics Documentation Error

[#1]

The enrichment configuration discussed in the this link seems to have an error:

{
  "index": "mad",
  "batchSize": 1,
  "enrichment": {
    "fieldMap": {
      "stellar" : {
        "config" : {
          "parser_score" : "OUTLIER_MAD_SCORE(OUTLIER_MAD_STATE_MERGE(
PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)"
         ,"is_alert" : "if parser_score > 3.5 then true else is_alert"
        }
      }
    }
  ,"fieldToTypeMap": { }
  },
  "threatIntel": {
    "fieldMap": { },
    "fieldToTypeMap": { },
    "triageConfig" : {
      "riskLevelRules" : [
        {
          "rule" : "parser_score > 3.5",
          "score" : 10
        }
      ],
      "aggregator" : "MAX"
    }
  }
}

I think the first two lines should appear in the corresponding indexing configuration and not the enrichment configuration. Using the enrichment config as is results in a parse error when pushing the configuration to zookeeper.

I just wanted to confirm this is the case. If not, what do those first two lines exactly mean?

[#2]

Hi @HS, Thank you for your comment on our Metron documentation. Sorry the delay in responding to you. I've raised your question to the Dev team and I hope to have an answer for you shortly.

[#3 · Accepted solution]

Hi guys. Yes, it would appear that doc example is outdated. "index" and "batchSize" belong in the indexing config. Here is a sample for bro from the current source:

cat metron-platform/metron-enrichment/src/main/config/zookeeper/enrichments/bro.json
{
  "enrichment" : {
    "fieldMap": {
      "geo": ["ip_dst_addr", "ip_src_addr"],
      "host": ["host"]
    }
  },
  "threatIntel": {
    "fieldMap": {
      "hbaseThreatIntel": ["ip_src_addr", "ip_dst_addr"]
    },
    "fieldToTypeMap": {
      "ip_src_addr" : ["malicious_ip"],
      "ip_dst_addr" : ["malicious_ip"]
    }
  }
}

cat metron-platform/metron-indexing/src/main/config/zookeeper/indexing/bro.json
{
  "hdfs" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "elasticsearch" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "solr" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : false
  }
}

[#4]

I submitted a fix to this README. You can view progress here

• https://github.com/apache/metron/pull/601
• https://issues.apache.org/jira/browse/METRON-977

@HS, the Metron community always looking for contributors and future committers and is extremely helpful (imho) in getting users involved. I see you've been active on the Metron boards here on HCC and we would be happy to get you open source community credit should you also choose to submit Jiras/PRs in the future. Best, Mike.