Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to start metron parser with HDP 2.6.5

Highlighted

Unable to start metron parser with HDP 2.6.5

Expert Contributor

Execute['/usr/hcp/1.6.0.0-7/metron/bin/start_parser_topology.sh -k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT'] {'logoutput': True, 'tries': 3, 'user': 'metron', 'try_sleep': 5} Submitting parser topology; args='-k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT' Running: /usr/jdk64/jdk1.8.0_112/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/hcp/1.6.0.0-7/metron/lib/metron-parsers-0.5.1.1.6.0.0-7-uber.jar /tmp/1deecdb80e6011e9b9db005056a4f2b7.jar Running: /usr/jdk64/jdk1.8.0_112/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar:/tmp/1deecdb80e6011e9b9db005056a4f2b7.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.6.5.0-292/storm/bin -Dstorm.jar=/tmp/1deecdb80e6011e9b9db005056a4f2b7.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.metron.parsers.topology.ParserTopologyCLI -k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT 628 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting 681 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED org.apache.metron.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "invalidWriterClassName" (class org.apache.metron.common.configuration.SensorParserConfig), not marked as ignorable (25 known properties: "errorWriterParallelism", "numAckers", "parserClassName", "readMetadata", "errorTopic", "errorWriterClassName", "filterClassName", "outputTopic", "numWorkers", "writerClassName", "rawMessageStrategy", "spoutConfig", "rawMessageStrategyConfig", "sensorTopic", "securityProtocol", "spoutNumTasks", "parserParallelism", "parserConfig", "errorWriterNumTasks", "cacheConfig", "mergeMetadata", "spoutParallelism", "fieldTransformations", "parserNumTasks", "stormConfig"]) at [Source: java.io.ByteArrayInputStream@33ecda92; line: 1, column: 192] (through reference chain: org.apache.metron.common.configuration.SensorParserConfig["invalidWriterClassName"]) at org.apache.metron.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62) at org.apache.metron.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:851) at org.apache.metron.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1085) at org.apache.metron.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1389) at org.apache.metron.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1367) at org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:266) at org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125) at org.apache.metron.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3807) at org.apache.metron.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2867) at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:102) at org.apache.metron.common.configuration.ParserConfigurations.updateSensorParserConfig(ParserConfigurations.java:40) at org.apache.metron.common.configuration.ParserConfigurations.updateSensorParserConfig(ParserConfigurations.java:36) at org.apache.metron.common.configuration.ConfigurationsUtils.lambda$updateParserConfigsFromZookeeper$0(ConfigurationsUtils.java:219) at org.apache.metron.common.configuration.ConfigurationsUtils.updateConfigsFromZookeeper(ConfigurationsUtils.java:209) at org.apache.metron.common.configuration.ConfigurationsUtils.updateParserConfigsFromZookeeper(ConfigurationsUtils.java:217) at org.apache.metron.parsers.topology.ParserTopologyBuilder.getSensorParserConfig(ParserTopologyBuilder.java:378) at org.apache.metron.parsers.topology.ParserTopologyBuilder.build(ParserTopologyBuilder.java:120) at org.apache.metron.parsers.topology.ParserTopologyCLI.getParserTopology(ParserTopologyCLI.java:571) at org.apache.metron.parsers.topology.ParserTopologyCLI.createParserTopology(ParserTopologyCLI.java:540) at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:601)

3 REPLIES 3

Re: Unable to start metron parser with HDP 2.6.5

New Contributor

check all your sensor configs....the top few lines are below. Remove the invalidwriterclassname and see if that solves the problem

"parserClassName": "org.apache.metron.parsers.bro.BasicBroParser", "filterClassName": null, "sensorTopic": "bro", "writerClassName": null, "errorWriterClassName": null, "invalidWriterClassName": null,

Re: Unable to start metron parser with HDP 2.6.5

Expert Contributor

Finally, parser started working..!!

I just cleared the parser zookeeper metadata directory and restarted the metron parser.

Re: Unable to start metron parser with HDP 2.6.5

New Contributor

how to clear parser zookeeper metadata directory or where