Created on 03-24-2019 10:10 PM - edited 09-16-2022 07:15 AM
Hi All,
Our Dev is running a Spark job and it needs to create a folder under '/hadoop/data11/yarn/nm/usercache/' with the user running the job. The path should be like '/hadoop/data11/yarn/nm/usercache/user1'. But i'm getting error as 'java.lang.IllegalArgumentException: No Config File/hadoop/data11/yarn/nm/usercache/user1/appcache/application_1549376940004_4169/container_1549376940004_4169_01_000001/hdfs:/user/user1/pqrstu/config/input_abc1234.xml'.
Please help.
Created 03-25-2019 08:09 AM
Hi ,
This looks like application is trying to find an hdfs file under yarn local directory
hdfs:/user/user1/pqrstu/config/input_abc1234.xml
The file being create here should be just input_abc1234.xml
Not sure what might be casuing this. Can you please give us the exact command you are using to submit spark job.
Thanks
Bimal