Spark 2.3 Csd failed to load

Question

Hello,

I am trying to install Spark 2.3.0 release 4, but after placing csd (jar file) to /opt/cloudera/csd i cannot add it from service.

I checked on this url http:xyz:7180/cmf/csd/refreshand got the following error:

placeholderHandlers":[],"invalidCsds":{"/opt/cloudera/csd/SPARK2_ON_YARN-2.3.0.cloudera4.jar":["Failed to load CSD from /opt/cloudera/csd/SPARK2_ON_YARN-2.3.0.cloudera4.jar, Error: CSD bundle data directory or Jar file doesn't exist"]},"repoEnabled":true}}

I followed these steps:

a. Download the spark 2.3.0 jar

b. Upload the CSD to /opt/cloudera/csd in the Cloudera Manager server.

c. Change the owner and group for the JAR

d. restart cloudera scm server and management service

Can anyone help ? I have CDH release 5.16.1-1.cdh5.16.1.p0.3

hadoopNoob · Accepted Answer

I have solve the issue thanks for your help

Bharati · Answer

I think you are missing the parcel installation along with CDS download. Whenever upgrading spark 2.x versions please follow all steps below and make sure there aren't any old version CSD in CSD folder A) Make sure the Parcel is installed distributed and Activated. Refer Step 3 in the document below: https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html B) Make sure you download Spark 2 CDS on Cloudera Manager Server host and delete old CDS https://www.cloudera.com/documentation/spark2/latest/topics/spark2_packaging.html#versions C) and followed steps mentioned in the link below https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_addon_services.html#concept_kpt_spj_bn - Installing the Custom Service Descriptor File - Acquire the CSD file from Cloudera or an ISV. - Log on to the Cloudera Manager Server host, and place the CSD file under the location configured for CSD files. - Set the file ownership to cloudera-scm:cloudera-scm with permission 644. - Restart the Cloudera Manager Server: service cloudera-scm-server restart - Log into the Cloudera Manager Admin Console and restart the Cloudera Management Service. Do one of the following: a) Select Clusters > Cloudera Management Service. Select Actions > Restart. b) On the Home > Status tab, click to the right of Cloudera Management Service and select Restart. - Click Restart to confirm. The Command Details window shows the progress of stopping and then starting the roles. - When Command completed with n/n successful subcommands appears, the task is complete. Click Close. - Then try to Add Service for Spark 2.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.