Reply
Contributor
Posts: 31
Registered: ‎03-02-2017
Accepted Solution

Log4j.properties permission denied

 

 

Dear All,

 

I am facing this issue in Cloudera 5.9. 5.11 and 5.12

 

when i am submitting a spark job through oozie this exception comes

 

 

/var/run/cloudera-scm-agent/process/578-yarn-NODEMANAGER/log4j.properties (permission denied)

 

 

The spark project is built with scala 2.10.6 and java 1.8.

 

regards

Shafi

Cloudera Employee
Posts: 10
Registered: ‎03-01-2016

Re: Log4j.properties permission denied

#1) Do you have spark gateway installed and client configuration deployed on this host?

#2) Do you have Spark selected as one of the dependent service on CM oozie configuration page?

 

 

Usgually the client should be able to source the log4j file from /etc/spark/conf/log4j.properties, instead of the NodeManager process dir.

Posts: 1,639
Kudos: 314
Solutions: 254
Registered: ‎07-31-2013

Re: Log4j.properties permission denied

If you're seeing this exception in the Oozie Spark action launcher log,
please ignore it as it may be expected in a secure environment. If your
launcher/action is truly failing, then the real exception will lie in the
logs that follow/other parts of the log - the log4j permission error can be
treated as a red herring as you investigate.
Contributor
Posts: 31
Registered: ‎03-02-2017

Re: Log4j.properties permission denied

Hi

 

Yes through oozie in kerberised env, it shows Spark job as FAILED.

 

Application Manager tries two attempts and it fails

 

but actually in Yarn logs i see the spark dataframe output and message as SUCCEEDED

 

regards

Shafi

New Contributor
Posts: 1
Registered: ‎02-20-2018

Re: Log4j.properties permission denied

[ Edited ]

The easiest workaround for this is to copy the log4j.propeties file to oozie.sharelib.path. You can even add all the extra files and jars which are required in the yarn container when scheduling via oozie.

Highlighted
Contributor
Posts: 31
Registered: ‎03-02-2017

Re: Log4j.properties permission denied

Hi

 

The issue was related to customized environment and it is fixed now.

 

regards

Shafi

Announcements