Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Logging Spark Jobs with log4j

Logging Spark Jobs with log4j

New Contributor


We need to log Spark jobs with log4j, saving the output into an external logfile in HDFS (we want to separate the application log from the native YARN logs).

We do not have Apache Flume and Apache Kafka on our cluster. (HDP 3) so we are looking for HDFS appender to use with log4j.

Any suggestions?

Thanks in advance.


Don't have an account?
Coming from Hortonworks? Activate your account here