Reply
Explorer
Posts: 10
Registered: ‎11-21-2017

Error starting JobHistoryServer on Cloudera Manager

[ Edited ]

I installed Cloudera Manager 5.13.

While installing and running for the first time YARN.. I had the following error
  

  Error starting JobHistoryServer
    org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done directory: [hdfs://vmi150132.contaboserver.net:8020/user/history/done]
        at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.tryCreatingHistoryDirs(HistoryFileManager.java:682)
        at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.createHistoryDirs(HistoryFileManager.java:618)
        at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:579)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:95)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:154)
        at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:229)
        at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:239)
    Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
        at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:279)
        at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:260)
        at org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:240)

[.... and other rows of Stack exception errors]


So basically the problem is the permission of the HDFS folders.

Something like:
`sudo -u hdfs hdfs dfs -chmod -R 777 /`
will fix the error.

But my question is... Wouldn't be unsafe? Why Cloudera doesn't handle this permissions? Was this handled well in the prrevious versions?

Master
Posts: 430
Registered: ‎07-01-2015

Re: Error starting JobHistoryServer on Cloudera Manager

Probably the problem is in the user under which you ran the YARN job. Has this user a home directory in HDFS?
Also is temp folder in HDFS correctly set? (/tmp)