Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

Spark 1.3.0 missing classes

avatar
Rising Star

 

I installed CDH 5.4 with cloudera manager from http://archive.cloudera.com/cdh5/parcels/5.4/ 

 

Now when i start the pyspark or spark-shell i get the Exception: "Exception in thread "main" java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/Module" 

 

This might be caused by librarries missing from the spark-assembly-1.3.0-cdh5.4.0-hadoop2.6.0-cdh5.4.0.jar

There is no com.fasterxml sub directory in the lib directory of the jar.

 

In the 1.3.1 assembly downloaded from the apache spark website there is a com.fasterxml lib directory and starting spark-shell and pyspark works perfect.

 

How do i replace the CDH 5.4 parcel spark version with the apache spark version (for hadoop 2.6)  is it enough if i just upload the new spark-assembly to the spark hdfs directory?

 

and is there someting wrong with the CDH 5.4 spark jar? 

 

thx for any assistance

Who agreed with this topic