Metron Alert UI: indexing error with elasticsearch

[#1]

Hi,

I already configure apache metron using hcp 1.7.1 with ambari 2.6.2 on bare metal (single node) os centos 7. The problem was the alert UI page does not working properly. There are data on the alert UI page but contains error log file "metron-rest.log".

nested: IllegalStateException[[nested] failed to find nested object under path [metron_alert]

The parser used suricata 'eve.json' file, that load into nifi and from nifi to kafka. The topology on storm are created the parser used ' JSONMap '. Based on error on log file, that nested on elasticsearch indexing. After follow this link

https://metron.apache.org/current-book/metron-platform/metron-elasticsearch/index.html

It solved the issue temporarily, but the error comes up again when elasticsearch create new index. Did i need create a template for solve this issue or any other fix?

Thanks.

[#2 · Accepted solution]

Hi @Amirul

Yes, you need to create a template. Best way of creating it, is using an existing one of the example parsers that are delivered with HCP/Metron and modifying it to fit your new parser.

You'd need at least a section in that template with:

      "metron_alert" : {
      "type" : "nested"
    }

Here, I've written a small blog post about what you need to take care about, when you create a template:

https://datahovel.com/2018/11/27/how-to-define-elastic-search-templates-for-apache-metron/

Here is the official documentation that describes that you need a template:

https://metron.apache.org/current-book/metron-platform/metron-elasticsearch/index.html

[#3]

Hi @Stefan Kupstaitis-Dunkler

Thanks for your answer and thanks for the link.