question spark history server error class not found in Archives of Support Questions (Read Only)

spark history server error class not found

Eran — Sat, 12 Mar 2016 04:18:18 GMT

I am looking at a customer installation that has a spark history server problem: Exception in thread "main" java.lang.ClassNotFoundException: org.apache.spark.deploy.yarn.history.YarnHistoryProvider at java.net.URLClassLoader$1.run(URLClassLoader.java:366) Looking at the jar it is missing the YarnHistoryService any ideas how to get around this on this version: spark_2_3_4_0_3485-master.noarch

Re: spark history server error class not found

aervits — Sat, 12 Mar 2016 05:14:26 GMT

@eorgad

please see this https://community.hortonworks.com/questions/8682/spark-history-server-startup-error.html

Re: spark history server error class not found

Eran — Sat, 12 Mar 2016 05:53:59 GMT

This is an upgraded cluster at one of our customers, I have no explanation to the missing class in the jar but I found that class in the pre upgraded jar. I was looking to make configuration changes - ended up swapping the jars - and the old one works !!

Re: spark history server error class not found

aervits — Sat, 12 Mar 2016 06:56:25 GMT

That certainly works but going forward wouldn't it cause problems?