Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Job History Server failed to start with Path /user already exists as dir; cannot create link here

Job History Server failed to start with Path /user already exists as dir; cannot create link here

New Contributor

I enabled nameservice federation and restart the services, got job history server failure with the following error messages in log:

If I run dfs command, always get the error message:

# sudo -u hdfs hdfs dfs -ls /
ls: Path /user already exists as dir; cannot create link here

 

I don't know what is the problem.

 

2019-04-19 04:23:39,089 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping JobHistoryServer metrics system...
2019-04-19 04:23:39,089 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobHistoryServer metrics system stopped.
2019-04-19 04:23:39,089 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobHistoryServer metrics system shutdown complete.
2019-04-19 04:23:39,090 ERROR org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting JobHistoryServer
java.lang.RuntimeException: org.apache.hadoop.fs.FileAlreadyExistsException: Path /user already exists as dir; cannot create link here
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:459)
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:480)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getDefaultFileContext(JobHistoryUtils.java:247)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.ensurePathInDefaultFileSystem(JobHistoryUtils.java:277)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.getConfiguredHistoryServerDoneDirPrefix(JobHistoryUtils.java:224)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.tryCreatingHistoryDirs(HistoryFileManager.java:671)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.createHistoryDirs(HistoryFileManager.java:630)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:591)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:97)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:150)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:226)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:236)
Caused by: org.apache.hadoop.fs.FileAlreadyExistsException: Path /user already exists as dir; cannot create link here
at org.apache.hadoop.fs.viewfs.InodeTree.createLink(InodeTree.java:325)
at org.apache.hadoop.fs.viewfs.InodeTree.<init>(InodeTree.java:578)
at org.apache.hadoop.fs.viewfs.ViewFs$1.<init>(ViewFs.java:212)
at org.apache.hadoop.fs.viewfs.ViewFs.<init>(ViewFs.java:212)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.fs.AbstractFileSystem.newInstance(AbstractFileSystem.java:135)
at org.apache.hadoop.fs.AbstractFileSystem.createFileSystem(AbstractFileSystem.java:173)
at org.apache.hadoop.fs.AbstractFileSystem.get(AbstractFileSystem.java:258)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:331)
at org.apache.hadoop.fs.FileContext$2.run(FileContext.java:328)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.fs.FileContext.getAbstractFileSystem(FileContext.java:328)
at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:454)
... 15 more
2019-04-19 04:23:39,093 INFO org.apache.hadoop.util.ExitUtil: Exiting with status -1: Error starting JobHistoryServer
2019-04-19 04:23:39,096 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: SHUTDOWN_MSG: