03-24-2019 10:10 PM
Our Dev is running a Spark job and it needs to create a folder under '/hadoop/data11/yarn/nm/usercache/' with the user running the job. The path should be like '/hadoop/data11/yarn/nm/usercache/user1'. But i'm getting error as 'java.lang.IllegalArgumentException: No Config File/hadoop/data11/yarn/nm/usercache/user1/appcache/application_1549376940004_4169/container_1549376940004_4169_01_000001/hdfs:/user/user1/pqrstu/config/input_abc1234.xml'.
03-25-2019 08:09 AM
This looks like application is trying to find an hdfs file under yarn local directory
The file being create here should be just input_abc1234.xml
Not sure what might be casuing this. Can you please give us the exact command you are using to submit spark job.