Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Unable to get Spark logs compressed


Unable to get Spark logs compressed

Super Collaborator

Hi Guys,


I'm trying to compress the spark logs:

I set spark.evenLog.compress=true at Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-defaults.conf


the weird that running spark-shell from anynode cross the cluster, i get the log compressed, but all the scheulred spark jobs through oozie isn't getting compressed.