Reply
Explorer
Posts: 6
Registered: ‎10-13-2016

Error in installing Spark2 on CDH 5.10 Virtual VM

Hi,

I am installing Spark2 on Virtual Box VM CDH 5.10.0 but when adding Spark2 service in Cloudera Manager getting error as :: ParcelTagUnsatisfiedException: No parcel provided required tags: set([u'cdh'])

 

I have followed the instructions from : https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html

 

I am getting error while executing step number 5 Add the Spark 2 service.

 

Here is the stacktrace from cloudera-scm-agent logs:

 

[26/May/2017 07:10:35 +0000] 9823 MainThread parcel INFO prepare_environment begin: {u'SPARK2': u'2.0.0.cloudera2-1.cdh5.7.0.p0.118100'}, [u'cdh'], [u'cdh-plugin', u'hdfs-plugin']
[26/May/2017 07:10:35 +0000] 9823 MainThread parcel INFO The following requested parcels are not available: {}
[26/May/2017 07:10:35 +0000] 9823 MainThread parcel INFO Obtained tags ['spark2', 'cdh-plugin'] for parcel SPARK2
[26/May/2017 07:10:35 +0000] 9823 MainThread agent ERROR Failed to activate {u'refresh_files': [], u'config_generation': 0, u'auto_restart': False, u'running': True, u'required_tags': [u'cdh'], u'one_off': True, u'special_file_info': [], u'group': u'hdfs', u'id': 54, u'status_links': {}, u'name': u'dfs-create-dir', u'extra_groups': [], u'run_generation': 1, u'start_timeout_seconds': None, u'environment': {u'HADOOP_AUDIT_LOGGER': u'INFO,RFAAUDIT', u'CM_ADD_TO_CP_DIRS': u'navigator/cdh57', u'FAST_MKDIR': u'true', u'HADOOP_NAMENODE_OPTS': u'-Xms52428800 -Xmx52428800 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/hdfs_hdfs-NAMENODE_pid{{PID}}.hprof -XX:OnOutOfMemoryError={{AGENT_COMMON_DIR}}/killparent.sh', u'HADOOP_SECURITY_LOGGER': u'INFO,RFAS', u'HADOOP_CREDSTORE_PASSWORD': u'bzprr93z5xatl2ckkjmmaojy4', u'HADOOP_LOG_DIR': u'/var/log/hadoop-hdfs', u'HADOOP_ROOT_LOGGER': u'INFO,console', u'HADOOP_LOGFILE': u'hadoop-cmf-hdfs-NAMENODE-quickstart.cloudera.log.out', u'CDH_VERSION': u'5'}, u'optional_tags': [u'cdh-plugin', u'hdfs-plugin'], u'program': u'hdfs/hdfs.sh', u'arguments': [u'mkdir', u'/user/spark', u'spark', u'spark', u'0751'], u'parcels': {u'SPARK2': u'2.0.0.cloudera2-1.cdh5.7.0.p0.118100'}, u'resources': [{u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': {u'limit_memlock': None, u'limit_fds': None}, u'file': None, u'memory': None, u'directory': None, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': True, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/hadoop-hdfs/stacks', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'hdfs', u'user': u'hdfs', u'mode': 493}, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/hadoop-hdfs', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'hadoop', u'user': u'hdfs', u'mode': 509}, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/lib/hadoop-hdfs', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'hadoop', u'user': u'hdfs', u'mode': 493}, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/lib/hadoop-hdfs/cache/hdfs/dfs/name', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'hadoop', u'user': u'hdfs', u'mode': 448}, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'192.168.56.101', u'port': 8020}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'192.168.56.101', u'port': 8022}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None, u'contents': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'192.168.56.101', u'port': 50070}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None, u'contents': None}], u'user': u'hdfs'}
Traceback (most recent call last):
File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/agent.py", line 1691, in handle_heartbeat_processes
new_process.update_heartbeat(raw, True)
File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/process.py", line 299, in update_heartbeat
self.fs_update()
File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/process.py", line 457, in fs_update
self.write_configuration(False)
File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/process.py", line 333, in write_configuration
self.update_process_environment_and_parcels()
File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/process.py", line 575, in update_process_environment_and_parcels
self.raw['optional_tags'])
File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.10.0-py2.6.egg/cmf/parcel.py", line 383, in prepare_environment
raise ParcelTagUnsatisfiedException("No parcel provided required tags: %s" % (missing_reqs,))
ParcelTagUnsatisfiedException: No parcel provided required tags: set([u'cdh'])

 

Any help will be appreciated.Thanks

 

Announcements