question Re: Failed to deploy Spark on CDH-5.3.0 in Archives of Support Questions (Read Only)

Failed to deploy Spark on CDH-5.3.0

Zhuo.Chen — Fri, 16 Sep 2022 09:17:23 GMT

CDH 5.3.0, unable to add Spark service, the webui display 'SparkUploadJarServiceCommand failed'.

check log file, found error as follows,

[Cannot find the assembly on local filesystem: /opt/cloudera/parcels/CDH-5.3.0-1.cdh5.3.0.p0.30/lib/spark/assembly/lib/spark-assembly*cdh*.jar]

To view the local filesystem, the Jar file does not exists indeed.

CDH5.2.x has no this problem cause the Jar file exists.

Re: Failed to deploy Spark on CDH-5.3.0

GautamG — Sun, 04 Jan 2015 09:08:53 GMT

This issue only occurs when you're using a Cloudera Manager version less
than 5.3.0 to manage a CDH 5.3.0 cluster. Please upgrade Cloudera Manager
as well to 5.3.0. If the error still persists, then run the following
commands to fix the symlink.

$ sudo mkdir -p -m 755 /opt/cloudera/parcels/CDH/lib/spark/assembly/lib
$ sudo ln -sf
/opt/cloudera/parcels/CDH/lib/spark/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar
/opt/cloudera/parcels/CDH/lib/spark/assembly/lib/spark-assembly-1.2.0-cdh5.3.0-hadoop2.5.0-cdh5.3.0.jar

Re: Failed to deploy Spark on CDH-5.3.0

Zhuo.Chen — Sun, 04 Jan 2015 09:21:49 GMT

You're totally right, thanks!

Re: Failed to deploy Spark on CDH-5.3.0

Zhuo.Chen — Sun, 04 Jan 2015 11:49:00 GMT

http://archive.cloudera.com/cm5/installer/5.2.1

http://archive.cloudera.com/cm5/installer/5.3.0

I download cm5 (v5.2.1 & v5.3.0) from above url,

however, I found the version is still v5.2.0 when I installed it.

Re: Failed to deploy Spark on CDH-5.3.0

GautamG — Sun, 04 Jan 2015 11:58:53 GMT

If you have Cloudera Manager already installed, please follow the upgrade
guide to move to v5.3.0.

http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_ag_upgrade_cm5.html

Re: Failed to deploy Spark on CDH-5.3.0

Zhuo.Chen — Sun, 04 Jan 2015 12:11:19 GMT

Thanks for your prompt reply. I'll try it.