Support Questions

Find answers, ask questions, and share your expertise

Exception in indexingBolt of indexing topology

avatar
Contributor

After runing Metron a little while,I received this exception:

index [bro_index_2017.02.23.16], type [bro_doc], id [AVpp_hu_luwdJ-LP4qUA], message [MapperParsingException[failed to parse [ip_dst_addr]]; nested: IllegalArgumentException[failed to parse ip [ff02::0001:0003], not a valid ipv4 address (4 dots)];]

How do i resolve it ? I'll appreciate it for any help!

1 ACCEPTED SOLUTION

avatar
Contributor

Well, the problem is actually in the elasticsearch indexing templates. Normally, I'd say that you could use a message filter to filter out the IPv6 data in the parser, but I know that they don't work in HCS 1.0. As a workaround, you could transform the IPv6 addresses to 0.0.0.0 and they'll index. You can also save off the old address in a new field. This would be how you would do it with Stellar field transformations.

Edit $METRON_HOME/config/zookeeper/parsers/bro.json to add the "fieldTransformations" section, like so:

{
  "parserClassName":"org.apache.metron.parsers.bro.BasicBroParser",
  "sensorTopic":"bro",
  "parserConfig": {},
  "fieldTransformations" : [
    {
      "transformation" : "STELLAR"
    ,"output" : [ "raw_dst_ip"
                , "ip_dst_addr"
                ]
    ,"config" : {
 "raw_dst_ip" : "ip_dst_addr"
,"ip_dst_addr" : "if IS_IP(ip_dst_addr, 'IPV4') then ip_dst_addr else '0.0.0.0'"
                }
    }
                           ]
}

If things work out like they should, you'll have a raw_dst_ip field and ip_dst_addr will either be IPv4 or '0.0.0.0', which will index just fine.

In the next release, you'll have a message filter that works so you could drop them easier.

Hope this helps! Report back if you get into trouble.

View solution in original post

11 REPLIES 11

avatar
Contributor

Thank you so much! I will feed back if it works.

avatar
Contributor

Thank you ! It works.