Created 02-24-2017 05:30 AM
After runing Metron a little while,I received this exception:
index [bro_index_2017.02.23.16], type [bro_doc], id [AVpp_hu_luwdJ-LP4qUA], message [MapperParsingException[failed to parse [ip_dst_addr]]; nested: IllegalArgumentException[failed to parse ip [ff02::0001:0003], not a valid ipv4 address (4 dots)];]
How do i resolve it ? I'll appreciate it for any help!
Created 02-24-2017 06:01 AM
Well, the problem is actually in the elasticsearch indexing templates. Normally, I'd say that you could use a message filter to filter out the IPv6 data in the parser, but I know that they don't work in HCS 1.0. As a workaround, you could transform the IPv6 addresses to 0.0.0.0 and they'll index. You can also save off the old address in a new field. This would be how you would do it with Stellar field transformations.
Edit $METRON_HOME/config/zookeeper/parsers/bro.json to add the "fieldTransformations" section, like so:
{ "parserClassName":"org.apache.metron.parsers.bro.BasicBroParser", "sensorTopic":"bro", "parserConfig": {}, "fieldTransformations" : [ { "transformation" : "STELLAR" ,"output" : [ "raw_dst_ip" , "ip_dst_addr" ] ,"config" : { "raw_dst_ip" : "ip_dst_addr" ,"ip_dst_addr" : "if IS_IP(ip_dst_addr, 'IPV4') then ip_dst_addr else '0.0.0.0'" } } ] }
If things work out like they should, you'll have a raw_dst_ip field and ip_dst_addr will either be IPv4 or '0.0.0.0', which will index just fine.
In the next release, you'll have a message filter that works so you could drop them easier.
Hope this helps! Report back if you get into trouble.
Created 02-28-2017 02:52 AM
Thank you so much! I will feed back if it works.
Created 02-28-2017 07:43 AM
Thank you ! It works.