Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Spark stanalone mode there is library issue ?

Contributor

Hi All,

When I am running the spark in local mode my code is working fine and when I am running it in standalone mode and deploy mode as client , I am getting the class not found exception "Caused by: java.lang.ClassNotFoundException:" for my custom class .Actually I am distributing the jar which has the class to worker by setting the property in spark configuration sparkConfig.setJars(new String[]{"myjar.jar"}); and I am seeing this jar getting distributed on worker node as well ,but I am getting the class not found exception.Why the executor is not able to fine the class in the jar on the worker node.

Could someone please let me what mistake I am making here .

Thanks in advance ,

Param.

2 REPLIES 2

Guru

@Param NC, Can you please try to add the necessary jar to --driver-class-path and --jars option while running application?

https://support.datastax.com/hc/en-us/articles/207442243-Spark-submit-fails-with-class-not-found-whe...

Contributor

@yvora

Actually I am submitting my spark application through program , I have done the same thing programatically by adding the sparkConfig.setJars(new String[]{"myjar.jar"}); still it is not working .

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.