Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Logging Spark Jobs with log4j

Highlighted

Logging Spark Jobs with log4j

New Contributor

Hi!

We need to log Spark jobs with log4j, saving the output into an external logfile in HDFS (we want to separate the application log from the native YARN logs).

We do not have Apache Flume and Apache Kafka on our cluster. (HDP 3) so we are looking for HDFS appender to use with log4j.

Any suggestions?

Thanks in advance.

R.A.