Reply
Cloudera Employee
Posts: 12
Registered: ‎03-31-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

Could you try stopping the agent, then creating a symlink at /etc/init.d which points to the script inside the tarball install? Something like: 

 

ln -s /<tarball-root>/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent

 

Then use "service cloudera-scm-agent start" to start it again. 

 

The other error could be related but it's difficult to say. 

Explorer
Posts: 14
Registered: ‎04-15-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

thanks a lot。but it diid work propery:[PRODUCT-root]#ln -s /opt/cm-5.3.3/etc/init.d/cloudera-scm-agent /etc/init.d/cloudera-scm-agent
[PRODUCT-root]#service cloudera-scm-agent start
File not found: /usr/sbin/cmf-agent
Explorer
Posts: 14
Registered: ‎04-15-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

I was successful install CM 5.1.3 in exact the same way of startup CM agent。is there another possibility ? for one,CDH bug or something like that
Explorer
Posts: 14
Registered: ‎04-15-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

thanks for you advice,and should i downgrade CDH from CM 5.3.3 to CM 5.2.4?
Cloudera Employee
Posts: 12
Registered: ‎03-31-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

Having looked at the error log again, it seems as though the problem is the env variable:

 

BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash $*`
}

 

Because it contains a line break, the string parsing is failing because it expects all env variables to contain '='. To confirm this, can you try running the agent without the BASH_FUNC_module env variable set?

Explorer
Posts: 14
Registered: ‎04-15-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

there is another error after commenting the following code from the script (/opt/cm-5.3.3/lib64/cmf/agent/src/cmf/util.py)。
[16/Apr/2015 00:42:34 +0000] 18649 MainThread agent ERROR Failed to activate {u'refresh_files': [u'cloudera-stack-monitor.
properties', u'capacity-scheduler.xml', u'fair-scheduler.xml', u'nodes_allow.txt', u'nodes_exclude.txt', u'topology.map'], u'config_
generation': 0, u'auto_restart': False, u'running': True, u'required_tags': [u'cdh'], u'user': u'yarn', u'special_file_info': [{u'pe
rmissions': 420, u'deploy_dir': u'/etc/hadoop/conf.cloudera.yarn', u'group': u'hadoop', u'name': u'topology.map', u'user': u'root'},
{u'permissions': 493, u'deploy_dir': u'/etc/hadoop/conf.cloudera.yarn', u'group': u'hadoop', u'name': u'topology.py', u'user': u'ro
ot'}], u'group': u'hadoop', u'id': 347, u'status_links': {u'status': u'http://hnfdpnn01:8088/'}, u'name': u'yarn-RESOURCEMANAGER', u
'run_generation': 1, u'environment': {u'YARN_RESOURCEMANAGER_OPTS': u'-Xms1073741824 -Xmx1073741824 -XX:+UseParNewGC -XX:+UseConcMar
kSweepGC -XX:-CMSConcurrentMTEnabled -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -Dhadoop.event.appender=,Ev
entCatcher -XX:OnOutOfMemoryError={{AGENT_COMMON_DIR}}/killparent.sh', u'YARN_LOGFILE': u'hadoop-cmf-yarn-RESOURCEMANAGER-hnfdpnn01.
log.out', u'YARN_ROOT_LOGGER': u'INFO,RFA', u'YARN_LOG_DIR': u'/var/log/hadoop-yarn', u'CDH_VERSION': u'5'}, u'optional_tags': [u'cd
h-plugin', u'yarn-plugin', u'mapreduce2-plugin'], u'program': u'yarn/yarn.sh', u'arguments': [u'resourcemanager'], u'parcels': {u'CD
H': u'5.3.3-1.cdh5.3.3.p0.5'}, u'resources': [{u'io': None, u'named_cpu': None, u'tcp_listen': None, u'dynamic': False, u'rlimits':
{u'limit_memlock': None, u'limit_fds': None}, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'nam
ed_cpu': None, u'tcp_listen': {u'bind_address': u'134.160.36.45', u'port': 8032}, u'dynamic': False, u'rlimits': None, u'file': None
, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'134.160.3
6.45', u'port': 8030}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io
': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'134.160.36.45', u'port': 8031}, u'dynamic': False, u'rlimits': None,
u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address
': u'134.160.36.45', u'port': 8033}, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': None, u'cpu'
: None}, {u'io': None, u'named_cpu': None, u'tcp_listen': {u'bind_address': u'134.160.36.45', u'port': 8088}, u'dynamic': False, u'r
limits': None, u'file': None, u'memory': None, u'directory': None, u'cpu': None}, {u'io': None, u'named_cpu': None, u'tcp_listen': N
one, u'dynamic': False, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/hadoop-yarn', u'bytes_f
ree_warning_threshhold_bytes': 0, u'group': u'hadoop', u'user': u'yarn', u'mode': 509}, u'cpu': None}, {u'io': None, u'named_cpu': N
one, u'tcp_listen': None, u'dynamic': True, u'rlimits': None, u'file': None, u'memory': None, u'directory': {u'path': u'/var/log/had
oop-yarn/stacks', u'bytes_free_warning_threshhold_bytes': 0, u'group': u'hadoop', u'user': u'yarn', u'mode': 493}, u'cpu': None}], u
'one_off': False}
Traceback (most recent call last):
File "/opt/cm-5.3.3/lib64/cmf/agent/src/cmf/agent.py", line 1234, in handle_heartbeat_response
new_process.activate()
File "/opt/cm-5.3.3/lib64/cmf/agent/src/cmf/agent.py", line 2441, in activate
self.write_process_conf()
File "/opt/cm-5.3.3/lib64/cmf/agent/src/cmf/agent.py", line 2551, in write_process_conf
"source_parcel_environment", env))
TypeError: 'NoneType' object is not iterable
Explorer
Posts: 14
Registered: ‎04-15-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

the commented code:
#try:
# return dict((line.split("=", 1) for line in data.splitlines()))
# except Exception, e:
# LOG.exception("Failed to parse environment variables: " + data)
# raise e
Cloudera Employee
Posts: 12
Registered: ‎03-31-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

Don't comment any code - please try removing the variable I mentioned from the shell environment as suggested. 

Explorer
Posts: 14
Registered: ‎04-15-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

how to start CM agent without the BASH_FUNC_module env variable set?
i took a try before starting CM agent with following command: source /etc/profile.d/modules.sh

it seems did't work.
Cloudera Employee
Posts: 12
Registered: ‎03-31-2015

Re: failed to start resourcemanager in the process of installing YARN with cdh5.3.3

You should be able to start it in the usual way by running the script in the tarball's etc/init.d directory  -- as far as I know there's no requirement for the BASH_FUNC_module var to be there for CM agent to work. Please post any error messages you see. 

Announcements