question Re: Log4j.properties permission denied in Archives of Support Questions (Read Only)

Log4j.properties permission denied

Shafiullah — Fri, 16 Sep 2022 12:08:43 GMT

Dear All,

I am facing this issue in Cloudera 5.9. 5.11 and 5.12

when i am submitting a spark job through oozie this exception comes

/var/run/cloudera-scm-agent/process/578-yarn-NODEMANAGER/log4j.properties (permission denied)

The spark project is built with scala 2.10.6 and java 1.8.

regards

Shafi

Re: Log4j.properties permission denied

Yuexin Zhang — Mon, 28 Aug 2017 07:03:56 GMT

#1) Do you have spark gateway installed and client configuration deployed on this host?

#2) Do you have Spark selected as one of the dependent service on CM oozie configuration page?

Usgually the client should be able to source the log4j file from /etc/spark/conf/log4j.properties, instead of the NodeManager process dir.

Re: Log4j.properties permission denied

Harsh J — Mon, 28 Aug 2017 10:15:11 GMT

If you're seeing this exception in the Oozie Spark action launcher log,
please ignore it as it may be expected in a secure environment. If your
launcher/action is truly failing, then the real exception will lie in the
logs that follow/other parts of the log - the log4j permission error can be
treated as a red herring as you investigate.

Re: Log4j.properties permission denied

Shafiullah — Wed, 13 Sep 2017 13:50:04 GMT

Yes through oozie in kerberised env, it shows Spark job as FAILED.

Application Manager tries two attempts and it fails

but actually in Yarn logs i see the spark dataframe output and message as SUCCEEDED

regards

Shafi

Re: Log4j.properties permission denied

Vinod254581 — Tue, 10 Apr 2018 09:08:48 GMT

The easiest workaround for this is to copy the log4j.propeties file to oozie.sharelib.path. You can even add all the extra files and jars which are required in the yarn container when scheduling via oozie.

Re: Log4j.properties permission denied

Shafiullah — Tue, 10 Apr 2018 09:10:15 GMT

The issue was related to customized environment and it is fixed now.

regards

Shafi

Re: Log4j.properties permission denied

Joselyn — Tue, 29 Jan 2019 16:45:11 GMT

Hello, I am facing the same problem, could you help me, and give me more detail, I will appreciate it.

Thanks in advance.