Created 02-24-2017 05:30 AM
After runing Metron a little while,I received this exception:
index [bro_index_2017.02.23.16], type [bro_doc], id [AVpp_hu_luwdJ-LP4qUA], message [MapperParsingException[failed to parse [ip_dst_addr]]; nested: IllegalArgumentException[failed to parse ip [ff02::0001:0003], not a valid ipv4 address (4 dots)];]
How do i resolve it ? I'll appreciate it for any help!
Created 02-24-2017 06:01 AM
Well, the problem is actually in the elasticsearch indexing templates. Normally, I'd say that you could use a message filter to filter out the IPv6 data in the parser, but I know that they don't work in HCS 1.0. As a workaround, you could transform the IPv6 addresses to 0.0.0.0 and they'll index. You can also save off the old address in a new field. This would be how you would do it with Stellar field transformations.
Edit $METRON_HOME/config/zookeeper/parsers/bro.json to add the "fieldTransformations" section, like so:
{
"parserClassName":"org.apache.metron.parsers.bro.BasicBroParser",
"sensorTopic":"bro",
"parserConfig": {},
"fieldTransformations" : [
{
"transformation" : "STELLAR"
,"output" : [ "raw_dst_ip"
, "ip_dst_addr"
]
,"config" : {
"raw_dst_ip" : "ip_dst_addr"
,"ip_dst_addr" : "if IS_IP(ip_dst_addr, 'IPV4') then ip_dst_addr else '0.0.0.0'"
}
}
]
}
If things work out like they should, you'll have a raw_dst_ip field and ip_dst_addr will either be IPv4 or '0.0.0.0', which will index just fine.
In the next release, you'll have a message filter that works so you could drop them easier.
Hope this helps! Report back if you get into trouble.
Created 02-28-2017 02:52 AM
Thank you so much! I will feed back if it works.
Created 02-28-2017 07:43 AM
Thank you ! It works.