Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search

Metron Enrichment Start

Labels:
rahul_perumandl
Explorer

Created on ‎07-10-2017 03:26 PM - edited ‎08-17-2019 05:20 PM

@Simon Elliston Ball

stderr: /var/lib/ambari-agent/data/errors-583.txt
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_master.py", line 113, in <module>
    Enrichment().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 314, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_master.py", line 74, in start
    commands.start_enrichment_topology()
  File "/var/lib/ambari-agent/cache/common-services/METRON/0.4.0.1.1.0.0/package/scripts/enrichment_commands.py", line 146, in start_enrichment_topology
    user=self.__params.metron_user)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
    self.env.run()
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 262, in action_run
    tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
    result = function(command, **kwargs)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
    tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
    result = _call(command, **kwargs_copy)
  File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/hcp/1.1.0.0-71/metron/bin/start_enrichment_topology.sh                                     -s enrichment                                     -z ip-10-0-1-45.ec2.internal:2181' returned 127. /usr/hcp/1.1.0.0-71/metron/bin/start_enrichment_topology.sh: line 22: storm: command not found
stdout: /var/lib/ambari-agent/data/output-583.txt
2017-07-10 15:11:58,099 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-07-10 15:11:58,245 - Stack Feature Version Info: stack_version=2.5, version=None, current_cluster_version=None -> 2.5
2017-07-10 15:11:58,252 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
User Group mapping (user_group) is missing in the hostLevelParams
2017-07-10 15:11:58,253 - Group['metron'] {}
2017-07-10 15:11:58,253 - Group['livy'] {}
2017-07-10 15:11:58,254 - Group['elasticsearch'] {}
2017-07-10 15:11:58,254 - Group['spark'] {}
2017-07-10 15:11:58,254 - Group['zeppelin'] {}
2017-07-10 15:11:58,254 - Group['hadoop'] {}
2017-07-10 15:11:58,254 - Group['kibana'] {}
2017-07-10 15:11:58,255 - Group['users'] {}
2017-07-10 15:11:58,255 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,256 - User['storm'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,256 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,257 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,257 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
2017-07-10 15:11:58,258 - User['zeppelin'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'zeppelin', u'hadoop']}
2017-07-10 15:11:58,258 - User['metron'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,259 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,259 - User['elasticsearch'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,260 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,260 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'users']}
2017-07-10 15:11:58,261 - User['flume'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,261 - User['kafka'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,262 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,262 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,263 - User['kibana'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,263 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,264 - User['hbase'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,265 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': [u'hadoop']}
2017-07-10 15:11:58,265 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-07-10 15:11:58,266 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-07-10 15:11:58,271 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-07-10 15:11:58,271 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'create_parents': True, 'mode': 0775, 'cd_access': 'a'}
2017-07-10 15:11:58,272 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-07-10 15:11:58,273 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2017-07-10 15:11:58,277 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2017-07-10 15:11:58,277 - Group['hdfs'] {}
2017-07-10 15:11:58,277 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': [u'hadoop', u'hdfs']}
2017-07-10 15:11:58,278 - FS Type: 
2017-07-10 15:11:58,278 - Directory['/etc/hadoop'] {'mode': 0755}
2017-07-10 15:11:58,290 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2017-07-10 15:11:58,291 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2017-07-10 15:11:58,304 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2017-07-10 15:11:58,312 - Skipping Execute[('setenforce', '0')] due to only_if
2017-07-10 15:11:58,312 - Directory['/var/log/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'hadoop', 'mode': 0775, 'cd_access': 'a'}
2017-07-10 15:11:58,314 - Directory['/var/run/hadoop'] {'owner': 'root', 'create_parents': True, 'group': 'root', 'cd_access': 'a'}
2017-07-10 15:11:58,314 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'create_parents': True, 'cd_access': 'a'}
2017-07-10 15:11:58,318 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2017-07-10 15:11:58,319 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2017-07-10 15:11:58,325 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2017-07-10 15:11:58,334 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs', 'group': 'hadoop'}
2017-07-10 15:11:58,334 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2017-07-10 15:11:58,335 - File['/usr/hdp/current/hadoop-client/conf/configuration.xsl'] {'owner': 'hdfs', 'group': 'hadoop'}
2017-07-10 15:11:58,339 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2017-07-10 15:11:58,342 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2017-07-10 15:11:58,514 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-07-10 15:11:58,515 - Running enrichment configure
2017-07-10 15:11:58,521 - File['/usr/hcp/1.1.0.0-71/metron/config/enrichment.properties'] {'owner': 'metron', 'content': Template('enrichment.properties.j2'), 'group': 'metron'}
2017-07-10 15:11:58,523 - Calling security setup
2017-07-10 15:11:58,524 - Create Metron Local Config Directory
2017-07-10 15:11:58,524 - Configure Metron global.json
2017-07-10 15:11:58,524 - Directory['/usr/hcp/1.1.0.0-71/metron/config/zookeeper'] {'owner': 'metron', 'group': 'metron', 'mode': 0755}
2017-07-10 15:11:58,526 - File['/usr/hcp/1.1.0.0-71/metron/config/zookeeper/global.json'] {'content': InlineTemplate(...), 'owner': 'metron'}
2017-07-10 15:11:58,528 - File['/usr/hcp/1.1.0.0-71/metron/config/zookeeper/../elasticsearch.properties'] {'content': InlineTemplate(...), 'owner': 'metron'}
2017-07-10 15:11:58,529 - Loading config into ZooKeeper
2017-07-10 15:11:58,530 - Execute['/usr/hcp/1.1.0.0-71/metron/bin/zk_load_configs.sh --mode PUSH -i /usr/hcp/1.1.0.0-71/metron/config/zookeeper -z ip-10-0-1-45.ec2.internal:2181'] {'path': [u'/usr/jdk64/jdk1.8.0_112/bin']}
2017-07-10 15:12:00,695 - Starting Metron enrichment topology: enrichment
2017-07-10 15:12:00,695 - Starting enrichment
2017-07-10 15:12:00,695 - Execute['/usr/hcp/1.1.0.0-71/metron/bin/start_enrichment_topology.sh                                     -s enrichment                                     -z ip-10-0-1-45.ec2.internal:2181'] {'user': 'metron'}

Command failed after 1 tries

16769-me.png

681 Views
0 Kudos
2 REPLIES 2

laurens
Explorer

Created ‎07-13-2017 09:33 PM

Which version is this? It complains that it can't find the command "storm" on line 22 in "start_enrichment_topology.sh". If I check on my machine, storm is at /usr/bin/storm...

532 Views
0 Kudos

rahul_perumandl
Explorer

Created ‎07-13-2017 10:23 PM

@Laurens Vets Thanks for replying. I figured out myself. The thing was the HDP package was not compatible with Metron. I was using an older version of HDP with newer version of Metron.

532 Views
0 Kudos
Take a Tour of the Community
Don't have an account?
Register
Your experience may be limited. Sign in to explore more.
Announcements
Product Announcements
[ANNOUNCE] CDP Private Cloud Data Services 1.5.0 Released
Community Announcements
January 2023 Community Highlights
Product Announcements
[ANNOUNCE] New Cloudera JDBC Connector 2.6.30 for Impala is Released
What's New @ Cloudera
Cloudera Operational Database (COD) provides a CLI option to enable HBase region canaries
What's New @ Cloudera
Cloudera Operational Database (COD) supports creating an operational database using a predefined Data Lake template
View More Announcements