Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Failed to deploy Spark on CDH-5.3.0

avatar
Contributor

CDH 5.3.0, unable to add Spark service, the webui display 'SparkUploadJarServiceCommand failed'.

 

check log file, found error as follows,

[Cannot find the assembly on local filesystem: /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark/assembly/lib/spark-assembly*cdh*.jar]

 

To view the local filesystem, the Jar file does not exists indeed.

 

CDH5.2.x has no this problem cause the Jar file exists.

 

1 ACCEPTED SOLUTION

avatar
If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Regards,
Gautam Gopalakrishnan

View solution in original post

5 REPLIES 5

avatar
This issue only occurs when you're using a Cloudera Manager version less
than 5.3.0 to manage a CDH 5.3.0 cluster. Please upgrade Cloudera Manager
as well to 5.3.0. If the error still persists, then run the following
commands to fix the symlink.

$ sudo mkdir -p -m 755 /opt/cloudera/parcels/CDH/lib/spark/assembly/lib
$ sudo ln -sf
/opt/cloudera/parcels/CDH/lib/spark/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar
/opt/cloudera/parcels/CDH/lib/spark/assembly/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar

Regards,
Gautam Gopalakrishnan

avatar
Contributor

You're totally right, thanks!

avatar
Contributor

http://archive.cloudera.com/cm5/installer/5.2.1

http://archive.cloudera.com/cm5/installer/5.3.0

 

I download cm5 (v5.2.1 & v5.3.0) from above url,

however, I found the version is still v5.2.0 when I installed it.

avatar
If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Regards,
Gautam Gopalakrishnan

avatar
Contributor

Thanks for your prompt reply. I'll try it.