Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

ashikinhtc — Sun, 18 Aug 2019 08:55:37 GMT

Hi, I am trying to install Metron Parser in Metron Service in Ambari but it is giving error. Please find below details of stderr.

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.2.2.0/package/scripts/parser_master.py", line 93, in <module>
    ParserMaster().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.2.2.0/package/scripts/parser_master.py", line 61, in start
    commands.start_parser_topologies(env)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.2.2.0/package/scripts/parser_commands.py", line 131, in start_parser_topologies
    user=self.__params.metron_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hcp/1.2.2.0-2/metron/bin/start_parser_topology.sh                                     -k sandbox.hortonworks.com:6667                                     -z sandbox.hortonworks.com:2181                                     -s bro                                     -ksp PLAINTEXT' returned 255. Running: /usr/lib/jvm/java/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.1.0-129/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.1.0-129/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.1.0-129/storm/lib/storm-rename-hack-1.1.0.2.6.1.0-129.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.1.0-129/storm/lib/zookeeper.jar:/usr/hdp/2.6.1.0-129/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.1.0-129/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.1.0-129/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.1.0-129/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.1.0-129/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.1.0-129/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.1.0-129/storm/lib/storm-core-1.1.0.2.6.1.0-129.jar:/usr/hdp/2.6.1.0-129/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.1.0-129/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.1.0-129/storm/lib/disruptor-3.3.2.jar org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/hcp/1.2.2.0-2/metron/lib/metron-parsers-0.4.1.1.2.2.0-2-uber.jar /tmp/0516d886954311e7bfe10242ac110002.jar
Running: /usr/lib/jvm/java/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.1.0-129/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.1.0-129/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.1.0-129/storm/lib/storm-rename-hack-1.1.0.2.6.1.0-129.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.1.0-129/storm/lib/zookeeper.jar:/usr/hdp/2.6.1.0-129/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.1.0-129/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.1.0-129/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.1.0-129/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.1.0-129/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.1.0-129/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.1.0-129/storm/lib/storm-core-1.1.0.2.6.1.0-129.jar:/usr/hdp/2.6.1.0-129/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.1.0-129/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.1.0-129/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.1.0-129/storm/lib/disruptor-3.3.2.jar:/tmp/0516d886954311e7bfe10242ac110002.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.6.1.0-129/storm/bin -Dstorm.jar=/tmp/0516d886954311e7bfe10242ac110002.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.metron.parsers.topology.ParserTopologyCLI -k sandbox.hortonworks.com:6667 -z sandbox.hortonworks.com:2181 -s bro -ksp PLAINTEXT
1163 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
1235 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
1699 [main] INFO  o.a.c.f.i.CuratorFrameworkImpl - Starting
1707 [main-EventThread] INFO  o.a.c.f.s.ConnectionStateManager - State change: CONNECTED
3111 [main] INFO  o.a.s.StormSubmitter - Generated ZooKeeper secret payload for MD5-digest: -7974437777870130871:-5056541602814490343
3190 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : sandbox.hortonworks.com:6627
3283 [main] INFO  o.a.s.s.a.AuthUtils - Got AutoCreds []
3288 [main] INFO  o.a.s.u.NimbusClient - Found leader nimbus : sandbox.hortonworks.com:6627
3300 [main] INFO  o.a.s.StormSubmitter - Uploading dependencies - jars...
3302 [main] INFO  o.a.s.StormSubmitter - Uploading dependencies - artifacts...
3302 [main] INFO  o.a.s.StormSubmitter - Dependency Blob keys - jars : [] / artifacts : []
3318 [main] INFO  o.a.s.StormSubmitter - Uploading topology jar /tmp/0516d886954311e7bfe10242ac110002.jar to assigned location: /hadoop/storm/nimbus/inbox/stormjar-2503662f-08bf-4778-a1de-60f848d48b2d.jar
4118 [main] INFO  o.a.s.StormSubmitter - Successfully uploaded topology jar to assigned location: /hadoop/storm/nimbus/inbox/stormjar-2503662f-08bf-4778-a1de-60f848d48b2d.jar
4118 [main] INFO  o.a.s.StormSubmitter - Submitting topology bro in distributed mode with conf {"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-7974437777870130871:-5056541602814490343"}
4608 [main] INFO  o.a.s.StormSubmitter - Finished submitting topology: bro
4609 [main] INFO  o.a.s.StormSubmitter - Initializing the registered ISubmitterHook [org.apache.atlas.storm.hook.StormAtlasHook]
4609 [main] WARN  o.a.s.StormSubmitter - Error occurred in invoking submitter hook:[org.apache.atlas.storm.hook.StormAtlasHook] 
java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[?:1.8.0_141]
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_141]
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335) ~[?:1.8.0_141]
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_141]
 at java.lang.Class.forName0(Native Method) ~[?:1.8.0_141]
 at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_141]
 at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]
 at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]
 at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]
 at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:162) [storm-core-1.1.0.2.6.1.0-129.jar:1.1.0.2.6.1.0-129]
 at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:334) [0516d886954311e7bfe10242ac110002.jar:?]
org.apache.storm.hooks.SubmitterHookException: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook
 at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:368)
 at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:278)
 at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:390)
 at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:162)
 at org.apache.metron.parsers.topology.ParserTopologyCLI.main(ParserTopologyCLI.java:334)
Caused by: java.lang.ClassNotFoundException: org.apache.atlas.storm.hook.StormAtlasHook
 at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:264)
 at org.apache.storm.StormSubmitter.invokeSubmitterHook(StormSubmitter.java:361)
 ... 4 more

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

jknulst — Wed, 13 Sep 2017 01:50:01 GMT

Ashikin,

Officially, all versions of Metron, including 1.2.2.0, have not been released to work with anything other then HDP-2.5.x

I am quite sure it does not work with HDP-2.6 since the Storm version is higher, which explains your error message. So you have to revert to a lower version of HDP.

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

jknulst — Mon, 02 Oct 2017 19:21:23 GMT

@Ashikin Abdullah please mark the question as answered if sufficiently answered.

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

jknulst — Mon, 02 Oct 2017 19:23:17 GMT

@Ashikin Abdullah HCP-1.3 / Metron 0.4.1.1.3.0.0-51 should work with HDP 2.6, but testing is still ongoing, so not for production usage.

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

ashikinhtc — Tue, 03 Oct 2017 08:33:00 GMT

Alright....Thanks Jasper 🙂

question Metron Parser Fail to start with Ambari 2.5 and HDP 2.6 in Archives of Support Questions (Read Only)

Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6

Re: Metron Parser Fail to start with Ambari 2.5 and HDP 2.6