Expert Contributor
Posts: 362
Registered: ‎01-25-2017

Unable to get Spark logs compressed

[ Edited ]

Hi Guys,


I'm trying to compress the spark logs:

I set spark.evenLog.compress=true at Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-defaults.conf


the weird that running spark-shell from anynode cross the cluster, i get the log compressed, but all the scheulred spark jobs through oozie isn't getting compressed.