Support Questions

Find answers, ask questions, and share your expertise

Spark job not creating the user folder under user cache while running the job


Hi All,


Our Dev is running a Spark job and it needs to create a folder under '/hadoop/data11/yarn/nm/usercache/' with the user running the job. The path should be like '/hadoop/data11/yarn/nm/usercache/user1'. But i'm getting error as 'java.lang.IllegalArgumentException: No Config File/hadoop/data11/yarn/nm/usercache/user1/appcache/application_1549376940004_4169/container_1549376940004_4169_01_000001/hdfs:/user/user1/pqrstu/config/input_abc1234.xml'. 


Please help.


Cloudera Employee

Hi ,


This looks like application is trying to find an hdfs file under yarn local directory




The file being create here should be just input_abc1234.xml


Not sure what might be casuing this. Can you please give us the exact command you are using to submit spark job.



Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.