Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Metron Statistics Documentation Error

Solved Go to solution

Metron Statistics Documentation Error

Explorer

The enrichment configuration discussed in the this link seems to have an error:

{
  "index": "mad",
  "batchSize": 1,
  "enrichment": {
    "fieldMap": {
      "stellar" : {
        "config" : {
          "parser_score" : "OUTLIER_MAD_SCORE(OUTLIER_MAD_STATE_MERGE(
PROFILE_GET( 'sketchy_mad', 'global', PROFILE_FIXED(10, 'MINUTES')) ), value)"
         ,"is_alert" : "if parser_score > 3.5 then true else is_alert"
        }
      }
    }
  ,"fieldToTypeMap": { }
  },
  "threatIntel": {
    "fieldMap": { },
    "fieldToTypeMap": { },
    "triageConfig" : {
      "riskLevelRules" : [
        {
          "rule" : "parser_score > 3.5",
          "score" : 10
        }
      ],
      "aggregator" : "MAX"
    }
  }
}

I think the first two lines should appear in the corresponding indexing configuration and not the enrichment configuration. Using the enrichment config as is results in a parse error when pushing the configuration to zookeeper.

I just wanted to confirm this is the case. If not, what do those first two lines exactly mean?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Metron Statistics Documentation Error

Explorer

Hi guys. Yes, it would appear that doc example is outdated. "index" and "batchSize" belong in the indexing config. Here is a sample for bro from the current source:

cat metron-platform/metron-enrichment/src/main/config/zookeeper/enrichments/bro.json
{
  "enrichment" : {
    "fieldMap": {
      "geo": ["ip_dst_addr", "ip_src_addr"],
      "host": ["host"]
    }
  },
  "threatIntel": {
    "fieldMap": {
      "hbaseThreatIntel": ["ip_src_addr", "ip_dst_addr"]
    },
    "fieldToTypeMap": {
      "ip_src_addr" : ["malicious_ip"],
      "ip_dst_addr" : ["malicious_ip"]
    }
  }
}
cat metron-platform/metron-indexing/src/main/config/zookeeper/indexing/bro.json
{
  "hdfs" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "elasticsearch" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "solr" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : false
  }
}

View solution in original post

3 REPLIES 3
Highlighted

Re: Metron Statistics Documentation Error

Explorer

Hi @HS, Thank you for your comment on our Metron documentation. Sorry the delay in responding to you. I've raised your question to the Dev team and I hope to have an answer for you shortly.

Highlighted

Re: Metron Statistics Documentation Error

Explorer

Hi guys. Yes, it would appear that doc example is outdated. "index" and "batchSize" belong in the indexing config. Here is a sample for bro from the current source:

cat metron-platform/metron-enrichment/src/main/config/zookeeper/enrichments/bro.json
{
  "enrichment" : {
    "fieldMap": {
      "geo": ["ip_dst_addr", "ip_src_addr"],
      "host": ["host"]
    }
  },
  "threatIntel": {
    "fieldMap": {
      "hbaseThreatIntel": ["ip_src_addr", "ip_dst_addr"]
    },
    "fieldToTypeMap": {
      "ip_src_addr" : ["malicious_ip"],
      "ip_dst_addr" : ["malicious_ip"]
    }
  }
}
cat metron-platform/metron-indexing/src/main/config/zookeeper/indexing/bro.json
{
  "hdfs" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "elasticsearch" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : true
  },
  "solr" : {
    "index": "bro",
    "batchSize": 5,
    "enabled" : false
  }
}

View solution in original post

Highlighted

Re: Metron Statistics Documentation Error

Explorer

I submitted a fix to this README. You can view progress here

@HS, the Metron community always looking for contributors and future committers and is extremely helpful (imho) in getting users involved. I see you've been active on the Metron boards here on HCC and we would be happy to get you open source community credit should you also choose to submit Jiras/PRs in the future. Best, Mike.

Don't have an account?
Coming from Hortonworks? Activate your account here