Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

CDH 5.5 pyspark java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

avatar
Contributor

Hello,

 

After upgrading a centos VM ( single node) from CDH 5.4.x  to CDH 5.5, pyspark fails to start

with the error

 

 java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

 

HADOOP_HOME is pointing to correct path in /etc/spark/conf/set-env.sh

 

Any  pointers on debugging this issue ? I dont want to download newer VM from cloudera since i have too much invested in the current VM

 

I used parcels to upgrade CDH5.4.x to CDH 5.5 via cloudera manager. 

Completed all the tasks specided to deploy new jars etc..

 

Who agreed with this topic