Support Questions
Find answers, ask questions, and share your expertise

Unable to start metron parser with HDP 2.6.5

Expert Contributor

Execute['/usr/hcp/1.6.0.0-7/metron/bin/start_parser_topology.sh -k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT'] {'logoutput': True, 'tries': 3, 'user': 'metron', 'try_sleep': 5} Submitting parser topology; args='-k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT' Running: /usr/jdk64/jdk1.8.0_112/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/hcp/1.6.0.0-7/metron/lib/metron-parsers-0.5.1.1.6.0.0-7-uber.jar /tmp/1deecdb80e6011e9b9db005056a4f2b7.jar Running: /usr/jdk64/jdk1.8.0_112/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar:/tmp/1deecdb80e6011e9b9db005056a4f2b7.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.6.5.0-292/storm/bin -Dstorm.jar=/tmp/1deecdb80e6011e9b9db005056a4f2b7.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.metron.parsers.topology.ParserTopologyCLI -k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT 628 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting 681 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED org.apache.metron.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "invalidWriterClassName" (class org.apache.metron.common.configuration.SensorParserConfig), not marked as ignorable (25 known properties: "errorWriterParallelism", "numAckers", "parserClassName", "readMetadata", "errorTopic", "errorWriterClassName", "filterClassName", "outputTopic", "numWorkers", "writerClassName", "rawMessageStrategy", "spoutConfig", "rawMessageStrategyConfig", "sensorTopic", "securityProtocol", "spoutNumTasks", "parserParallelism", "parserConfig", "errorWriterNumTasks", "cacheConfig", "mergeMetadata", "spoutParallelism", "fieldTransformations", "parserNumTasks", "stormConfig"]) at [Source: java.io.ByteArrayInputStream@33ecda92; line: 1, column: 192] (through reference chain: org.apache.metron.common.configuration.SensorParserConfig["invalidWriterClassName"]) at org.apache.metron.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62) at org.apache.metron.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:851) at org.apache.metron.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1085) at org.apache.metron.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1389) at org.apache.metron.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1367) at org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:266) at org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125) at org.apache.metron.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3807) at org.apache.metron.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2867) at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:102) at org.apache.metron.common.configuration.ParserConfigurations.updateSensorParserConfig(ParserConfigurations.java:40) at org.apache.metron.common.configuration.ParserConfigurations.updateSensorParserConfig(ParserConfigurations.java:36) at org.apache.metron.common.configuration.ConfigurationsUtils.lambda$updateParserConfigsFromZookeeper$0(ConfigurationsUtils.java:219) at org.apache.metron.common.configuration.ConfigurationsUtils.updateConfigsFromZookeeper(ConfigurationsUtils.java:209) at org.apache.metron.common.configuration.ConfigurationsUtils.updateParserConfigsFromZookeeper(ConfigurationsUtils.java:217) at org.apache.metron.parsers.topology.ParserTopologyBuilder.getSensorParserConfig(ParserTopologyBuilder.java:378) at org.apache.metron.parsers.topology.ParserTopologyBuilder.build(ParserTopologyBuilder.java:120) at org.apache.metron.parsers.topology.ParserTopologyCLI.getParserTopology(ParserTopologyCLI.java:571) at org.apache.metron.parsers.topology.ParserTopologyCLI.createParserTopology(ParserTopologyCLI.java:540) at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:601)

3 REPLIES 3

Explorer

check all your sensor configs....the top few lines are below. Remove the invalidwriterclassname and see if that solves the problem

"parserClassName": "org.apache.metron.parsers.bro.BasicBroParser", "filterClassName": null, "sensorTopic": "bro", "writerClassName": null, "errorWriterClassName": null, "invalidWriterClassName": null,

Expert Contributor

Finally, parser started working..!!

I just cleared the parser zookeeper metadata directory and restarted the metron parser.

how to clear parser zookeeper metadata directory or where