Hello community,
I have deployed Apache Metron in a single node Vagrant VM. Everything works as expected so far (I'm able to read Bro, Snort, Yaf sensors, enrich the data and index it). However, now I have Suricata data streaming into Metron. Suricata data is being queue at Metron's Kafka for parsing. Furthermore, I have a Logstash that filters Suricata data and writes JSON messages to Kafka.
See an example of a Kafka message (as inputing to the parser topology)
{"in_packets":17,"data_type":"suricata_event","in_bytes":13055,"dvc_type":"suricata","conn_state":"closed","out_packets":17,"@version":"1","id":"9dcca116-86e5-4044-b033-8bdb7053bd7e","ip_dst_addr":"10.5.0.14","nproto":"TCP","raw":"{\"tcp\":{\"fin\":true,\"tcp_flags_tc\":\"1b\",\"ack\":true,\"syn\":true,\"tcp_flags\":\"1b\",\"psh\":true,\"state\":\"closed\",\"tcp_flags_ts\":\"1b\"},\"proto\":\"TCP\",\"source\":\"/var/log/suricata/eve.json\",\"data_type\":\"suricata_event\",\"tags\":[\"beats_input_codec_json_applied\"],\"dest_port\":6690,\"dvc_type\":\"suricata\",\"beat\":{\"name\":\"redbox1\",\"version\":\"6.2.3\",\"hostname\":\"redbox1\"},\"@version\":\"1\",\"dvc_host\":\"redbox1\",\"src_ip\":\"10.5.0.148\",\"src_port\":46560,\"timestamp\":\"2018-04-11T12:44:22.003733+0000\",\"@timestamp\":\"2018-04-11T12:55:05.147Z\",\"prospector\":{\"type\":\"log\"},\"app_proto\":\"failed\",\"offset\":20805212,\"dest_ip\":\"10.5.0.14\",\"host\":\"redbox1\",\"flow_id\":2078884479188050,\"event_type\":\"flow\",\"flow\":{\"bytes_toserver\":4001,\"age\":0,\"end\":\"2018-04-11T12:43:21.821489+0000\",\"start\":\"2018-04-11T12:43:21.670802+0000\",\"reason\":\"timeout\",\"alerted\":false,\"bytes_toclient\":13055,\"pkts_toclient\":17,\"state\":\"closed\",\"pkts_toserver\":17}}","additional_atts":"{\"reason\":\"timeout\"}","duration":0,"ip_src_port":46560,"ip_dst_port":6690,"@timestamp":"2018-04-12T12:21:10.192Z","ip_src_addr":"10.5.0.148","eventid":2078884479188050,"out_bytes":4001,"event_type":"flow","dvc_time":1523450662003,"eventtime":1523450601670,"aproto":"failed"}Then, I have defined my "suricata" Sensor in Metron's Management UI using a "JSONMap" Parser and a sample message is correctly parsed.
When I try to start the suricata sensor, it does not start correctly. I tried both using the Metron Management UI and the command line:
[vagrant@node1 /]$ sudo sh usr/metron/0.4.3/bin/start_parser_topology.sh -k $KAFKA_HOST:6667 -z $ZOOKEEPER_HOST:2181 -s suricata
After this, I get this error:
Running: /usr/jdk64/jdk1.8.0_112/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.4.0-91/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.4.0-91/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.4.0-91/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.4.0-91/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.4.0-91/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.4.0-91/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/storm-core-1.1.0.2.6.4.0-91.jar:/usr/hdp/2.6.4.0-91/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.4.0-91/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.4.0-91/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.4.0-91/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.4.0-91/storm/lib/storm-rename-hack-1.1.0.2.6.4.0-91.jar:/usr/hdp/2.6.4.0-91/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.4.0-91/storm/lib/zookeeper.jar:/usr/hdp/2.6.4.0-91/storm/lib/minlog-1.3.0.jar org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/metron/0.4.3/lib/metron-parsers-0.4.3-uber.jar /tmp/077916523e5311e8b7ea525400e0ddcb.jar
Running: /usr/jdk64/jdk1.8.0_112/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.4.0-91/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.4.0-91/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.4.0-91/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.4.0-91/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.4.0-91/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.4.0-91/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/storm-core-1.1.0.2.6.4.0-91.jar:/usr/hdp/2.6.4.0-91/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.4.0-91/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.4.0-91/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.4.0-91/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.4.0-91/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.4.0-91/storm/lib/storm-rename-hack-1.1.0.2.6.4.0-91.jar:/usr/hdp/2.6.4.0-91/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.4.0-91/storm/lib/zookeeper.jar:/usr/hdp/2.6.4.0-91/storm/lib/minlog-1.3.0.jar:/tmp/077916523e5311e8b7ea525400e0ddcb.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.6.4.0-91/storm/bin -Dstorm.jar=/tmp/077916523e5311e8b7ea525400e0ddcb.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.metron.parsers.topology.ParserTopologyCLI -k :6667 -z :2181 -s suricata
993 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting
1105 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
1803 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting
1806 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
java.lang.IllegalStateException: Bolt 'parserBolt' contains a non-serializable field of type org.apache.metron.parsers.json.JSONMapParser$1, which was instantiated prior to topology creation. org.apache.metron.parsers.json.JSONMapParser$1 should be instantiated within the prepare method of 'parserBolt at the earliest.
at org.apache.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:128)
at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:459)
Caused by: java.lang.RuntimeException: java.io.NotSerializableException: org.apache.metron.parsers.json.JSONMapParser$1
at org.apache.storm.utils.Utils.javaSerialize(Utils.java:238)
at org.apache.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:123)
... 1 more
Caused by: java.io.NotSerializableException: org.apache.metron.parsers.json.JSONMapParser$1
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
at org.apache.storm.utils.Utils.javaSerialize(Utils.java:234)
... 2 more
