Created on 08-05-2016 04:10 AM - edited 09-16-2022 03:33 AM
Using Cloudera Manager for CDH 5.8.0, deploying on AWS machines with RH 7.2.
When installing YARN, the NodeManagers fail to start. Looking at the logs, there is an exception as follows:
Service org.apache.hadoop.yarn.server.nodemanager.recovery.NMLeveldbStateStoreService failed in state INITED; cause: org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state/LOCK: Permission denied org.fusesource.leveldbjni.internal.NativeDB$DBException: IO error: /var/lib/hadoop-yarn/yarn-nm-recovery/yarn-nm-state/LOCK: Permission denied at org.fusesource.leveldbjni.internal.NativeDB.checkStatus(NativeDB.java:200) at org.fusesource.leveldbjni.internal.NativeDB.open(NativeDB.java:218) at org.fusesource.leveldbjni.JniDBFactory.open(JniDBFactory.java:168) at ....
This error is actually identical to the one reported in this post
Although the version/diagnostics are not the same.
Anyway, I've connected to one of NodeManager machines to take a look, and eventually I've seen that the home directory of yarn belonged to root and did not have any permissions:
d---------. 2 root root 6 Aug 5 05:53 hadoop-yarn
As soon as I've given permissions to it, the NodeManager has started properly.
chown -R yarn:yarn /var/lib/hadoop-yarn chmod 770 /var/lib/hadoop-yarn drwxrwx---. 3 yarn yarn 49 Aug 5 06:37 hadoop-yarn
However, I don't want to do this for all the machines (and potentially for many other users that I see are wrong in the /var/lib directory), and I think it must be my mistake when installing...
I've tried removing the YARN service and installing it again, but I get the same result.
Thanks for your help.