Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Metron Statistics Documentation Error

avatar
Contributor

The enrichment configuration discussed in the this link seems to have an error:

{
  "index": "mad",
  "batchSize": 1,
  "enrichment": {
    "fieldMap": {
      "stellar" : {
        "config" : {
          "parser_score" : "OUTLIER_MAD_SCORE(OUTLIER_MAD_STATE_MERGE(
PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)"
         ,"is_alert" : "if parser_score > 3.5 then true else is_alert"
        }
      }
    }
  ,"fieldToTypeMap": { }
  },
  "threatIntel": {
    "fieldMap": { },
    "fieldToTypeMap": { },
    "triageConfig" : {
      "riskLevelRules" : [
        {
          "rule" : "parser_score > 3.5",
          "score" : 10
        }
      ],
      "aggregator" : "MAX"
    }
  }
}

I think the first two lines should appear in the corresponding indexing configuration and not the enrichment configuration. Using the enrichment config as is results in a parse error when pushing the configuration to zookeeper.

I just wanted to confirm this is the case. If not, what do those first two lines exactly mean?

1 ACCEPTED SOLUTION

avatar
Contributor

Hi guys. Yes, it would appear that doc example is outdated. "index" and "batchSize" belong in the indexing config. Here is a sample for bro from the current source:

cat metron-platform/metron-enrichment/src/main/config/zookeeper/enrichments/bro.json
{
  "enrichment" : {
    "fieldMap": {
      "geo": ["ip_dst_addr", "ip_src_addr"],
      "host": ["host"]
    }
  },
  "threatIntel": {
    "fieldMap": {
      "hbaseThreatIntel": ["ip_src_addr", "ip_dst_addr"]
    },
    "fieldToTypeMap": {
      "ip_src_addr" : ["malicious_ip"],
      "ip_dst_addr" : ["malicious_ip"]
    }
  }
}
cat metron-platform/metron-indexing/src/main/config/zookeeper/indexing/bro.json
{
  "hdfs" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "elasticsearch" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "solr" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : false
  }
}

View solution in original post

3 REPLIES 3

avatar
Contributor

Hi @HS, Thank you for your comment on our Metron documentation. Sorry the delay in responding to you. I've raised your question to the Dev team and I hope to have an answer for you shortly.

avatar
Contributor

Hi guys. Yes, it would appear that doc example is outdated. "index" and "batchSize" belong in the indexing config. Here is a sample for bro from the current source:

cat metron-platform/metron-enrichment/src/main/config/zookeeper/enrichments/bro.json
{
  "enrichment" : {
    "fieldMap": {
      "geo": ["ip_dst_addr", "ip_src_addr"],
      "host": ["host"]
    }
  },
  "threatIntel": {
    "fieldMap": {
      "hbaseThreatIntel": ["ip_src_addr", "ip_dst_addr"]
    },
    "fieldToTypeMap": {
      "ip_src_addr" : ["malicious_ip"],
      "ip_dst_addr" : ["malicious_ip"]
    }
  }
}
cat metron-platform/metron-indexing/src/main/config/zookeeper/indexing/bro.json
{
  "hdfs" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "elasticsearch" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "solr" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : false
  }
}

avatar
Contributor

I submitted a fix to this README. You can view progress here

@HS, the Metron community always looking for contributors and future committers and is extremely helpful (imho) in getting users involved. I see you've been active on the Metron boards here on HCC and we would be happy to get you open source community credit should you also choose to submit Jiras/PRs in the future. Best, Mike.