Created 09-12-2018 02:47 AM
metron-profiler-service-not-starting.txt
metron profiler startup fails with this error:
stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.0.0/package/scripts/profiler_master.py", line 112, in <module>
Profiler().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.0.0/package/scripts/profiler_master.py", line 89, in start
commands.start_profiler_topology(env)
File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.1.1.4.0.0/package/scripts/profiler_commands.py", line 142, in start_profiler_topology
Execute(start_cmd, user=self.__params.metron_user, tries=3, try_sleep=5, logoutput=True)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 262, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 303, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hcp/1.4.0.0-38/metron/bin/start_profiler_topology.sh -s profiler -z hdp01-ib.hpc.lan:2181,edge01-ib.hpc.lan:2181,hdp02-ib.hpc.lan:2181' returned 1. Running: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/jre/bin/java -server -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar org.apache.storm.daemon.ClientJarTransformerRunner org.apache.storm.hack.StormShadeTransformer /usr/hcp/1.5.0.0-9/metron/lib/metron-profiler-0.4.3.1.5.0.0-9-uber.jar /tmp/67f3c702ab9011e8b2e200215e5dc6c6.jar
Running: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.131-11.b12.el7.x86_64/jre/bin/java -Ddaemon.name= -Dstorm.options= -Dstorm.home=/usr/hdp/2.6.5.0-292/storm -Dstorm.log.dir=/var/log/storm -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/hdp/current/storm-client/lib -Dstorm.conf.file= -cp /usr/hdp/2.6.5.0-292/storm/lib/asm-5.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/clojure-1.7.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/disruptor-3.3.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/kryo-3.0.3.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-api-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-core-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/hdp/2.6.5.0-292/storm/lib/log4j-slf4j-impl-2.8.2.jar:/usr/hdp/2.6.5.0-292/storm/lib/minlog-1.3.0.jar:/usr/hdp/2.6.5.0-292/storm/lib/objenesis-2.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/reflectasm-1.10.1.jar:/usr/hdp/2.6.5.0-292/storm/lib/ring-cors-0.1.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/servlet-api-2.5.jar:/usr/hdp/2.6.5.0-292/storm/lib/slf4j-api-1.7.21.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-core-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/storm-rename-hack-1.1.0.2.6.5.0-292.jar:/usr/hdp/2.6.5.0-292/storm/lib/zookeeper.jar:/usr/hdp/2.6.5.0-292/storm/lib/ambari-metrics-storm-sink.jar:/tmp/67f3c702ab9011e8b2e200215e5dc6c6.jar:/usr/hdp/current/storm-supervisor/conf:/usr/hdp/2.6.5.0-292/storm/bin -Dstorm.jar=/tmp/67f3c702ab9011e8b2e200215e5dc6c6.jar -Dstorm.dependency.jars= -Dstorm.dependency.artifacts={} org.apache.storm.flux.Flux --remote /usr/hcp/1.5.0.0-9/metron/flux/profiler/remote.yaml --filter /usr/hcp/1.5.0.0-9/metron/config/profiler.properties
¦¦¦¦¦¦¦+¦¦+ ¦¦+ ¦¦+¦¦+ ¦¦+
¦¦+----+¦¦¦ ¦¦¦ ¦¦¦+¦¦+¦¦++
¦¦¦¦¦+ ¦¦¦ ¦¦¦ ¦¦¦ +¦¦¦++
¦¦+--+ ¦¦¦ ¦¦¦ ¦¦¦ ¦¦+¦¦+
¦¦¦ ¦¦¦¦¦¦¦++¦¦¦¦¦¦++¦¦++ ¦¦+
+-+ +------+ +-----+ +-+ +-+
+- Apache Storm -+
+- data FLow User eXperience -+
Version: 1.0.1
Parsing file: /usr/hcp/1.5.0.0-9/metron/flux/profiler/remote.yaml
932 [main] INFO o.a.s.f.p.FluxParser - loading YAML from input stream...
944 [main] INFO o.a.s.f.p.FluxParser - Performing property substitution.
954 [main] INFO o.a.s.f.p.FluxParser - Not performing environment variable substitution.
Exception in thread "main" while parsing a flow sequence
in 'string', line 165, column 21:
args: [${profiler.max.routes.per.bolt}]
^
expected ',' or ']', but got FlowMappingStart
in 'string', line 165, column 23:
args: [${profiler.max.routes.per.bolt}]
^
at org.yaml.snakeyaml.parser.ParserImpl$ParseFlowSequenceEntry.produce(ParserImpl.java:635)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:158)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:143)
at org.yaml.snakeyaml.composer.Composer.composeSequenceNode(Composer.java:203)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:157)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:237)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:159)
at org.yaml.snakeyaml.composer.Composer.composeSequenceNode(Composer.java:204)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:157)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:237)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:159)
at org.yaml.snakeyaml.composer.Composer.composeSequenceNode(Composer.java:204)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:157)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:237)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:159)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:122)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseConstructor.java:120)
at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:481)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:400)
at org.apache.storm.flux.parser.FluxParser.loadYaml(FluxParser.java:121)
at org.apache.storm.flux.parser.FluxParser.parseInputStream(FluxParser.java:75)
at org.apache.storm.flux.parser.FluxParser.parseFile(FluxParser.java:49)
at org.apache.storm.flux.Flux.runCli(Flux.java:145)
at org.apache.storm.flux.Flux.main(Flux.java:98)
Created 09-12-2018 02:52 AM
This HCC thread looks like a duplicate of the other one: https://community.hortonworks.com/questions/221774/metron-rest-service-start-failed.html?childToView...
.
Can you please close one of them so that all in a single HCC threads everyone can share their findings.
I am just pasting my response on the other thread here:
As we see the cause of failure then we find the following :
Parsing file: /usr/hcp/1.5.0.0-9/metron/flux/profiler/remote.yaml 932 [main] INFO o.a.s.f.p.FluxParser - loading YAML from input stream... 944 [main] INFO o.a.s.f.p.FluxParser - Performing property substitution. 954 [main] INFO o.a.s.f.p.FluxParser - Not performing environment variable substitution. Exception in thread "main" while parsing a flow sequence in 'string', line 165, column 21: args: [${profiler.max.routes.per.bolt}] ^ expected ',' or ']', but got FlowMappingStart in 'string', line 165, column 23: args: [${profiler.max.routes.per.bolt}]
.
Above seems to be a YAML parsing error which might be due to incorrect data. So can you please check the YAML file "/usr/hcp/1.5.0.0-9/metron/flux/profiler/remote.yaml" file using some Online YAML parsing tool and then validate to see if there are some special characters which is causing the parsing failure. Specially around the property "profiler.max.routes.per.bolt". Looks like there may be some issue at line 165 on this file.
After fixing the value please restart it. If you still face any issue then please share the file.
.
Created 09-12-2018 01:28 PM
this one can be closed, how do I close it?