Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Restart Resource Manager does not work, can not start YARN

Solved Go to solution
Highlighted

Restart Resource Manager does not work, can not start YARN

New Contributor

I set up successfully the Ambari server but some sevices can not be started.

If I want o start or restart the resource manager it aborts (after some time) giving me following message:

2018-11-19 00:13:30,067 - get_user_call_output returned (0, u'{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File does not exist: /ats/done/"}}404', u'')
2018-11-19 00:13:30,068 - Will retry 2 time(s), caught exception: DFS directory '/ats/done/' does not exist !. Sleeping for 20 sec(s)

It says /ats/done is not existing. Where do I create this folder?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Restart Resource Manager does not work, can not start YARN

@Andreas Kühnert,

I guess the directory got deleted somehow from HDFS. You can create it and try starting RM

# su hdfs
# kinit -kt /etc/security/keytabs/hdfs.headless.keytab {principal}  ---> Run this if your environment is kerberized
# hdfs dfs -mkdir -p /ats/done
# hdfs dfs -chown -R yarn:hadoop /ats/done

After running the commands, try to restart RM. If it fails with /ats/active directory not found, repeat the same steps changing the directory name.

.

If this works , please take a moment to login and "Accept" the answer.

4 REPLIES 4

Re: Restart Resource Manager does not work, can not start YARN

@Andreas Kühnert,

I guess the directory got deleted somehow from HDFS. You can create it and try starting RM

# su hdfs
# kinit -kt /etc/security/keytabs/hdfs.headless.keytab {principal}  ---> Run this if your environment is kerberized
# hdfs dfs -mkdir -p /ats/done
# hdfs dfs -chown -R yarn:hadoop /ats/done

After running the commands, try to restart RM. If it fails with /ats/active directory not found, repeat the same steps changing the directory name.

.

If this works , please take a moment to login and "Accept" the answer.

Re: Restart Resource Manager does not work, can not start YARN

New Contributor

Hello it worked.

But now I am facing another issue. When I am starting YARN Registry DNS it says:

 File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/3.0.0.0-1634/hadoop/libexec && /usr/hdp/3.0.0.0-1634/hadoop-yarn/bin/yarn --config /usr/hdp/3.0.0.0-1634/hadoop/conf --daemon start registrydns' returned 1. mesg: ttyname failed: Inappropriate ioctl for device
ERROR: Cannot set priority of registrydns process 24309

I already set ulimit to 10000 as recommended.

Whats the issue here?

Re: Restart Resource Manager does not work, can not start YARN

New Contributor

Okay it worked!


But now I am facing issue with starting YARN Registry DNS.

 File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/3.0.0.0-1634/hadoop/libexec && /usr/hdp/3.0.0.0-1634/hadoop-yarn/bin/yarn --config /usr/hdp/3.0.0.0-1634/hadoop/conf --daemon start registrydns' returned 1. mesg: ttyname failed: Inappropriate ioctl for device
ERROR: Cannot set priority of registrydns process 24309

Whats the issue here? Why it can not set its priority?

Thanks in advance ;)

Re: Restart Resource Manager does not work, can not start YARN

@Andreas Kühnert,

Glad that the issue is resolved. Since this is a different issue, I suggest to open a new thread for this issue so that the main thread doesn't get deviated. I'm not sure of that issue, may be other experts can help :)

Don't have an account?
Coming from Hortonworks? Activate your account here