Failing Oozie-Spark with core-site.xml (Permission denied)

[#1]

Hello all,

I am getting Permission denied error below when launching Oozie-Spark workflow. Also the directory with read, write access

/opt/data/yarn/local/usercache/oozie/appcache

drwxrwxrwx 2 yarn hadoop 10 Jul 20 11:30 appcache

Could anyone please help me with below error? I really appreciated. Thanks

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied)

java.io.FileNotFoundException: /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied)

[#2]

@JT Ng

As the error itself says, the oozie user is not able to access the file most probably.

So, check the permission on the entire path to the file, not just the very last folder in the file path.

[#3]

Hi @nkumar

I copied core-site.xml file into app path. and give permission as group hdfs and user hdfs as I am running SparkAction as hdfs user.

-rw-r--r--   3 hdfs hdfs       3371 2018-06-07 07:27 /user/hdfs/sparkoozie/TransformationOper.jar
-rw-r--r--   3 hdfs hdfs        774 2018-06-07 07:27 /user/hdfs/sparkoozie/core-site.xml
-rw-r--r--   3 hdfs hdfs        390 2018-06-07 07:27 /user/hdfs/sparkoozie/job.properties
-rw-r--r--   3 hdfs hdfs        923 2018-06-07 07:27 /user/hdfs/sparkoozie/workflow.xml

But still I am getting above error.

Regards,

Jay.

[#4]

@JT Ng Oozie spark action is not supported instead use spark oozie shell action.