Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to start metron parser with HDP 2.6.5

Unable to start metron parser with HDP 2.6.5

Expert Contributor

Execute['/usr/hcp/1.6.0.0-7/metron/bin/start_parser_topology.sh -k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT'] {'logoutput': True, 'tries': 3, 'user': 'metron', 'try_sleep': 5} Submitting parser topology; args='-k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT' Running: /usr/jdk64/jdk1.8.0_112/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/hcp/1.6.0.0-7/metron/lib/metron-parsers-0.5.1.1.6.0.0-7-uber.jar /tmp/1deecdb80e6011e9b9db005056a4f2b7.jar Running: /usr/jdk64/jdk1.8.0_112/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar:/tmp/1deecdb80e6011e9b9db005056a4f2b7.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.6.5.0-292/storm/bin -Dstorm.jar=/tmp/1deecdb80e6011e9b9db005056a4f2b7.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.metron.parsers.topology.ParserTopologyCLI -k <server>:6667 -z <server>:2181 -s bro -ksp PLAINTEXT 628 [main] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting 681 [main-EventThread] INFO o.a.c.f.s.ConnectionStateManager - State change: CONNECTED org.apache.metron.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "invalidWriterClassName" (class org.apache.metron.common.configuration.SensorParserConfig), not marked as ignorable (25 known properties: "errorWriterParallelism", "numAckers", "parserClassName", "readMetadata", "errorTopic", "errorWriterClassName", "filterClassName", "outputTopic", "numWorkers", "writerClassName", "rawMessageStrategy", "spoutConfig", "rawMessageStrategyConfig", "sensorTopic", "securityProtocol", "spoutNumTasks", "parserParallelism", "parserConfig", "errorWriterNumTasks", "cacheConfig", "mergeMetadata", "spoutParallelism", "fieldTransformations", "parserNumTasks", "stormConfig"]) at [Source: java.io.ByteArrayInputStream@33ecda92; line: 1, column: 192] (through reference chain: org.apache.metron.common.configuration.SensorParserConfig["invalidWriterClassName"]) at org.apache.metron.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:62) at org.apache.metron.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:851) at org.apache.metron.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1085) at org.apache.metron.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1389) at org.apache.metron.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1367) at org.apache.metron.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:266) at org.apache.metron.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:125) at org.apache.metron.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3807) at org.apache.metron.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2867) at org.apache.metron.common.utils.JSONUtils.load(JSONUtils.java:102) at org.apache.metron.common.configuration.ParserConfigurations.updateSensorParserConfig(ParserConfigurations.java:40) at org.apache.metron.common.configuration.ParserConfigurations.updateSensorParserConfig(ParserConfigurations.java:36) at org.apache.metron.common.configuration.ConfigurationsUtils.lambda$updateParserConfigsFromZookeeper$0(ConfigurationsUtils.java:219) at org.apache.metron.common.configuration.ConfigurationsUtils.updateConfigsFromZookeeper(ConfigurationsUtils.java:209) at org.apache.metron.common.configuration.ConfigurationsUtils.updateParserConfigsFromZookeeper(ConfigurationsUtils.java:217) at org.apache.metron.parsers.topology.ParserTopologyBuilder.getSensorParserConfig(ParserTopologyBuilder.java:378) at org.apache.metron.parsers.topology.ParserTopologyBuilder.build(ParserTopologyBuilder.java:120) at org.apache.metron.parsers.topology.ParserTopologyCLI.getParserTopology(ParserTopologyCLI.java:571) at org.apache.metron.parsers.topology.ParserTopologyCLI.createParserTopology(ParserTopologyCLI.java:540) at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:601)

3 REPLIES 3
Highlighted

Re: Unable to start metron parser with HDP 2.6.5

Explorer

check all your sensor configs....the top few lines are below. Remove the invalidwriterclassname and see if that solves the problem

"parserClassName": "org.apache.metron.parsers.bro.BasicBroParser", "filterClassName": null, "sensorTopic": "bro", "writerClassName": null, "errorWriterClassName": null, "invalidWriterClassName": null,

Highlighted

Re: Unable to start metron parser with HDP 2.6.5

Expert Contributor

Finally, parser started working..!!

I just cleared the parser zookeeper metadata directory and restarted the metron parser.

Highlighted

Re: Unable to start metron parser with HDP 2.6.5

how to clear parser zookeeper metadata directory or where


Don't have an account?
Coming from Hortonworks? Activate your account here