Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Failing Oozie-Spark with core-site.xml (Permission denied)

Failing Oozie-Spark with core-site.xml (Permission denied)

New Contributor

Hello all,

I am getting Permission denied error below when launching Oozie-Spark workflow. Also the directory with read, write access

/opt/data/yarn/local/usercache/oozie/appcache

drwxrwxrwx 2 yarn hadoop 10 Jul 20 11:30 appcache

Could anyone please help me with below error? I really appreciated. Thanks

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied)

java.io.FileNotFoundException: /opt/data/yarn/local/usercache/oozie/appcache/application_1500495991334_1431/container_e04_1500495991334_1431_01_000002/core-site.xml (Permission denied)
3 REPLIES 3

Re: Failing Oozie-Spark with core-site.xml (Permission denied)

Cloudera Employee

@JT Ng

As the error itself says, the oozie user is not able to access the file most probably.

So, check the permission on the entire path to the file, not just the very last folder in the file path.

Re: Failing Oozie-Spark with core-site.xml (Permission denied)

Contributor

Hi @nkumar

I copied core-site.xml file into app path. and give permission as group hdfs and user hdfs as I am running SparkAction as hdfs user.

-rw-r--r--   3 hdfs hdfs       3371 2018-06-07 07:27 /user/hdfs/sparkoozie/TransformationOper.jar
-rw-r--r--   3 hdfs hdfs        774 2018-06-07 07:27 /user/hdfs/sparkoozie/core-site.xml
-rw-r--r--   3 hdfs hdfs        390 2018-06-07 07:27 /user/hdfs/sparkoozie/job.properties
-rw-r--r--   3 hdfs hdfs        923 2018-06-07 07:27 /user/hdfs/sparkoozie/workflow.xml

But still I am getting above error.

Regards,

Jay.

Highlighted

Re: Failing Oozie-Spark with core-site.xml (Permission denied)

New Contributor

@JT Ng Oozie spark action is not supported instead use spark oozie shell action.