Reply
Contributor
Posts: 30
Registered: ‎03-02-2017

Log4j.properties permission denied

 

 

Dear All,

 

I am facing this issue in Cloudera 5.9. 5.11 and 5.12

 

when i am submitting a spark job through oozie this exception comes

 

 

/var/run/cloudera-scm-agent/process/578-yarn-NODEMANAGER/log4j.properties (permission denied)

 

 

The spark project is built with scala 2.10.6 and java 1.8.

 

regards

Shafi

Cloudera Employee
Posts: 8
Registered: ‎03-01-2016

Re: Log4j.properties permission denied

#1) Do you have spark gateway installed and client configuration deployed on this host?

#2) Do you have Spark selected as one of the dependent service on CM oozie configuration page?

 

 

Usgually the client should be able to source the log4j file from /etc/spark/conf/log4j.properties, instead of the NodeManager process dir.

Posts: 1,565
Kudos: 286
Solutions: 239
Registered: ‎07-31-2013

Re: Log4j.properties permission denied

If you're seeing this exception in the Oozie Spark action launcher log,
please ignore it as it may be expected in a secure environment. If your
launcher/action is truly failing, then the real exception will lie in the
logs that follow/other parts of the log - the log4j permission error can be
treated as a red herring as you investigate.
Backline Customer Operations Engineer
Highlighted
Contributor
Posts: 30
Registered: ‎03-02-2017

Re: Log4j.properties permission denied

Hi

 

Yes through oozie in kerberised env, it shows Spark job as FAILED.

 

Application Manager tries two attempts and it fails

 

but actually in Yarn logs i see the spark dataframe output and message as SUCCEEDED

 

regards

Shafi

Announcements