Reply
Highlighted
Expert Contributor
Posts: 338
Registered: ‎01-25-2017

Unable to get Spark logs compressed

[ Edited ]

Hi Guys,

 

I'm trying to compress the spark logs:

I set spark.evenLog.compress=true at Spark Client Advanced Configuration Snippet (Safety Valve) for spark-conf/spark-defaults.conf

 

the weird that running spark-shell from anynode cross the cluster, i get the log compressed, but all the scheulred spark jobs through oozie isn't getting compressed.

 

 

Announcements