Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark Jar caching?

Highlighted

Spark Jar caching?

Contributor

I'm running an application with spark-submit. The application uses both Scala and Java. The spark-submit specifies the location of the jar file with --jars

 

A strange phenomenon I'm seeing - even though I make modifications to my Java files and build new jar files, the cluster sometimes uses my older jar files. It is as if the cluster has a cached copy of my old jar file. 

 

Can someone please educate me on where to look for older or cached jar files and clean them up?

 

ps: I'm using Cloudera 5.5.1, with Spark 1.5.0

 

Thanks.