I see below JSCH jar in the /etc/spark/conf.cloudera.spark_on_yarn/classpath.txt
I want to use latest jar (jsch-0.1.54.jar) of JSCH for my project. Could you please let me know how I can achieve that?
I have tried below in my spark-submit command. But still it is using the older (jsch-0.1.42.jar).
Also, please let me know if there is a possibility to have the latest jsch jar in the /etc/spark/conf.cloudera.spark_on_yarn/classpath.txt
Given that JSCH is apparently used internally in Spark, I think you'd probably have to use shading to use a different version in your project.
I tried shading and also I tried to create uber jar. But both cases, did not work. It was taking the older version which is there in the classpath.txt