Support Questions

Find answers, ask questions, and share your expertise

Failed to deploy Spark on CDH-5.3.0

avatar
Contributor

CDH 5.3.0, unable to add Spark service, the webui display 'SparkUploadJarServiceCommand failed'.

 

check log file, found error as follows,

[Cannot find the assembly on local filesystem: /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark/assembly/lib/spark-assembly*cdh*.jar]

 

To view the local filesystem, the Jar file does not exists indeed.

 

CDH5.2.x has no this problem cause the Jar file exists.

 

1 ACCEPTED SOLUTION

avatar
If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Regards,
Gautam Gopalakrishnan

View solution in original post

5 REPLIES 5

avatar
This issue only occurs when you're using a Cloudera Manager version less
than 5.3.0 to manage a CDH 5.3.0 cluster. Please upgrade Cloudera Manager
as well to 5.3.0. If the error still persists, then run the following
commands to fix the symlink.

$ sudo mkdir -p -m 755 /opt/cloudera/parcels/CDH/lib/spark/assembly/lib
$ sudo ln -sf
/opt/cloudera/parcels/CDH/lib/spark/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar
/opt/cloudera/parcels/CDH/lib/spark/assembly/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar

Regards,
Gautam Gopalakrishnan

avatar
Contributor

You're totally right, thanks!

avatar
Contributor

http://archive.cloudera.com/cm5/installer/5.2.1

http://archive.cloudera.com/cm5/installer/5.3.0

 

I download cm5 (v5.2.1 & v5.3.0) from above url,

however, I found the version is still v5.2.0 when I installed it.

avatar
If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Regards,
Gautam Gopalakrishnan

avatar
Contributor

Thanks for your prompt reply. I'll try it.