Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Failed to deploy Spark on CDH-5.3.0

avatar
Contributor

CDH 5.3.0, unable to add Spark service, the webui display 'SparkUploadJarServiceCommand failed'.

 

check log file, found error as follows,

[Cannot find the assembly on local filesystem: /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark/assembly/lib/spark-assembly*cdh*.jar]

 

To view the local filesystem, the Jar file does not exists indeed.

 

CDH5.2.x has no this problem cause the Jar file exists.

 

1 ACCEPTED SOLUTION

avatar
If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Regards,
Gautam Gopalakrishnan

View solution in original post

5 REPLIES 5

avatar
This issue only occurs when you're using a Cloudera Manager version less
than 5.3.0 to manage a CDH 5.3.0 cluster. Please upgrade Cloudera Manager
as well to 5.3.0. If the error still persists, then run the following
commands to fix the symlink.

$ sudo mkdir -p -m 755 /opt/cloudera/parcels/CDH/lib/spark/assembly/lib
$ sudo ln -sf
/opt/cloudera/parcels/CDH/lib/spark/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar
/opt/cloudera/parcels/CDH/lib/spark/assembly/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar

Regards,
Gautam Gopalakrishnan

avatar
Contributor

You're totally right, thanks!

avatar
Contributor

http://archive.cloudera.com/cm5/installer/5.2.1

http://archive.cloudera.com/cm5/installer/5.3.0

 

I download cm5 (v5.2.1 & v5.3.0) from above url,

however, I found the version is still v5.2.0 when I installed it.

avatar
If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Regards,
Gautam Gopalakrishnan

avatar
Contributor

Thanks for your prompt reply. I'll try it.