Support Questions

Find answers, ask questions, and share your expertise

Override JSCH libraries for spark

New Contributor

Hi Team,


I see below JSCH jar in the /etc/spark/conf.cloudera.spark_on_yarn/classpath.txt



I want to use latest jar (jsch-0.1.54.jar) of JSCH for my project. Could you please let me know how I can achieve that?


I have tried below in my spark-submit command. But still it is using the older (jsch-0.1.42.jar).

--conf 'spark.{driver,executor}.userClassPathFirst=true'

--conf 'spark.{driver,executor}.extraClassPath=../lib/jsch.jar'

--jars ../lib/jsch.jar


Also, please let me know if there is a possibility to have the latest jsch jar in the /etc/spark/conf.cloudera.spark_on_yarn/classpath.txt


Cloudera Employee

Given that JSCH is apparently used internally in Spark, I think you'd probably have to use shading to use a different version in your project.

New Contributor

I have tried shading also but did NOT work.

Cloudera Employee

What happened?

New Contributor

I tried shading and also I tried to create uber jar. But both cases, did not work. It was taking the older version which is there in the classpath.txt