Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Restart Resource Manager does not work, can not start YARN

avatar

I set up successfully the Ambari server but some sevices can not be started.

If I want o start or restart the resource manager it aborts (after some time) giving me following message:

2018-11-19 00:13:30,067 - get_user_call_output returned (0, u'{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File does not exist: /ats/done/"}}404', u'')
2018-11-19 00:13:30,068 - Will retry 2 time(s), caught exception: DFS directory '/ats/done/' does not exist !. Sleeping for 20 sec(s)

It says /ats/done is not existing. Where do I create this folder?

1 ACCEPTED SOLUTION

avatar
Super Guru

@Andreas Kühnert,

I guess the directory got deleted somehow from HDFS. You can create it and try starting RM

# su hdfs
# kinit -kt /etc/security/keytabs/hdfs.headless.keytab {principal}  ---> Run this if your environment is kerberized
# hdfs dfs -mkdir -p /ats/done
# hdfs dfs -chown -R yarn:hadoop /ats/done

After running the commands, try to restart RM. If it fails with /ats/active directory not found, repeat the same steps changing the directory name.

.

If this works , please take a moment to login and "Accept" the answer.

View solution in original post

4 REPLIES 4

avatar
Super Guru

@Andreas Kühnert,

I guess the directory got deleted somehow from HDFS. You can create it and try starting RM

# su hdfs
# kinit -kt /etc/security/keytabs/hdfs.headless.keytab {principal}  ---> Run this if your environment is kerberized
# hdfs dfs -mkdir -p /ats/done
# hdfs dfs -chown -R yarn:hadoop /ats/done

After running the commands, try to restart RM. If it fails with /ats/active directory not found, repeat the same steps changing the directory name.

.

If this works , please take a moment to login and "Accept" the answer.

avatar

Hello it worked.

But now I am facing another issue. When I am starting YARN Registry DNS it says:

 File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/3.0.0.0-1634/hadoop/libexec && /usr/hdp/3.0.0.0-1634/hadoop-yarn/bin/yarn --config /usr/hdp/3.0.0.0-1634/hadoop/conf --daemon start registrydns' returned 1. mesg: ttyname failed: Inappropriate ioctl for device
ERROR: Cannot set priority of registrydns process 24309

I already set ulimit to 10000 as recommended.

Whats the issue here?

avatar

Okay it worked!


But now I am facing issue with starting YARN Registry DNS.

 File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
    raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/hdp/3.0.0.0-1634/hadoop/libexec && /usr/hdp/3.0.0.0-1634/hadoop-yarn/bin/yarn --config /usr/hdp/3.0.0.0-1634/hadoop/conf --daemon start registrydns' returned 1. mesg: ttyname failed: Inappropriate ioctl for device
ERROR: Cannot set priority of registrydns process 24309

Whats the issue here? Why it can not set its priority?

Thanks in advance 😉

avatar
Super Guru

@Andreas Kühnert,

Glad that the issue is resolved. Since this is a different issue, I suggest to open a new thread for this issue so that the main thread doesn't get deviated. I'm not sure of that issue, may be other experts can help 🙂