Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark 2.3 Csd failed to load

Solved Go to solution

Spark 2.3 Csd failed to load

Rising Star

Hello,

 

I am trying to install Spark 2.3.0 release 4, but after placing csd (jar file) to /opt/cloudera/csd i cannot add it from service. 

I checked on this url http:xyz:7180/cmf/csd/refreshand got the following error:

 

placeholderHandlers":[],"invalidCsds":{"/opt/cloudera/csd/SPARK2_ON_YARN-2.3.0.cloudera4.jar":["Failed to load CSD from /opt/cloudera/csd/SPARK2_ON_YARN-2.3.0.cloudera4.jar, Error: CSD bundle data directory or Jar file doesn't exist"]},"repoEnabled":true}}

I followed these steps:

a. Download the spark 2.3.0 jar 

b. Upload the CSD to /opt/cloudera/csd in the Cloudera Manager server.

c. Change the owner and group for the JAR

d. restart cloudera scm server and management service

 

Can anyone help ? I have CDH release 5.16.1-1.cdh5.16.1.p0.3

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Spark 2.3 Csd failed to load

Rising Star
I have solve the issue thanks for your help
17 REPLIES 17

Re: Spark 2.3 Csd failed to load

Cloudera Employee
I think you are missing the parcel installation along with CDS download.

Whenever upgrading spark 2.x versions please follow all steps below and make sure there aren't any old version CSD in CSD folder A) Make sure the Parcel is installed distributed and Activated. Refer Step 3 in the document below: https://www.cloudera.com/documentation/spark2/latest/topics/spark2_installing.html B) Make sure you download Spark 2 CDS on Cloudera Manager Server host and delete old CDS https://www.cloudera.com/documentation/spark2/latest/topics/spark2_packaging.html#versions C) and followed steps mentioned in the link below https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_addon_services.html#concept_kp... - Installing the Custom Service Descriptor File - Acquire the CSD file from Cloudera or an ISV. - Log on to the Cloudera Manager Server host, and place the CSD file under the location configured for CSD files. - Set the file ownership to cloudera-scm:cloudera-scm with permission 644. - Restart the Cloudera Manager Server: service cloudera-scm-server restart - Log into the Cloudera Manager Admin Console and restart the Cloudera Management Service. Do one of the following: a) Select Clusters > Cloudera Management Service. Select Actions > Restart. b) On the Home > Status tab, click to the right of Cloudera Management Service and select Restart. - Click Restart to confirm. The Command Details window shows the progress of stopping and then starting the roles. - When Command completed with n/n successful subcommands appears, the task is complete. Click Close. - Then try to Add Service for Spark 2.

Re: Spark 2.3 Csd failed to load

Rising Star
I have followed exact steps but it still fails

Re: Spark 2.3 Csd failed to load

Rising Star
is it because of cloudera version ?

Re: Spark 2.3 Csd failed to load

Cloudera Employee

I see that you are on CDH 5.16. Spark 2.3 is compatible with that. What is the CM version you are on?

 

- Can you confirm if you have installed Spark 2.3 parcel in CM > Hosts > Parcels?

- Please post the directory permission for following:

ls -ltr /opt/cloudera/

 

ls -ltr /opt/cloudera/csd

 

 

 

 

Highlighted

Re: Spark 2.3 Csd failed to load

Rising Star

I have mentioned in the question as well

 

Version: Cloudera Express 5.16.1 (#1 built by jenkins on 20181120-1759 git: 6a13b87a6fcdf4afad6d4474a68a9434b24d6c67)

Java VM Name: Java HotSpot(TM) 64-Bit Server VM

Java Version: 1.8.0_191

Re: Spark 2.3 Csd failed to load

Cloudera Employee

I see that you are on CDH 5.16. Spark 2.3 is compatible with that. What is the CM version you are on?

 

- Can you confirm if you have installed Spark 2.3 parcel in CM > Hosts > Parcels? Send us the screenshot of the entire page if possible.

 

- Also please post the directory permission for following:

ls -ltr /opt/cloudera/

 

ls -ltr /opt/cloudera/csd

Re: Spark 2.3 Csd failed to load

Rising Star

Capture.PNG

 

Details on permissions:

 

ls -ltr /opt/cloudera/csd

total 20

-rw-r--r-- 1 cloudera-scm cloudera-scm 19037 Jan 11 23:04 SPARK2_ON_YARN-2.3.0.cloudera4.jar

 

ls -ltr /opt/cloudera/


total 16
drw-r--r-- 2 cloudera-scm cloudera-scm 4096 Jan 11 23:04 csd
drwxr-xr-x 2 cloudera-scm cloudera-scm 4096 Jan 11 23:09 parcel-repo
drwxr-xr-x 2 root root 4096 Jan 11 23:09 parcel-cache
drwxr-xr-x 5 root root 4096 Jan 11 23:09 parcels

 

Version: Cloudera Express 5.16.1 (#1 built by jenkins on 20181120-1759 git: 6a13b87a6fcdf4afad6d4474a68a9434b24d6c67)

Java VM Name: Java HotSpot(TM) 64-Bit Server VM

Java Version: 1.8.0_191

 

Re: Spark 2.3 Csd failed to load

Cloudera Employee

1) The error that you showed talks about SPARK2_ON_YARN-2.3.0.cloudera2.jar not SPARK2_ON_YARN-2.3.0.cloudera4.jar. These are 2 different versions.

ERROR 1704117900@scm-web-7:com.cloudera.csd.components.C                                                                                        sdLocalRepository: Failed to load CSD from /opt/cloudera/csd/SPARK2_ON_YARN-2.3.                                                                                        0.cloudera2.jar, Error: CSD bundle data directory or Jar file doesn't exist

 

2) Do you multiple CSDs in /opt/clodera/csd or parcels activated? If that's the case please deactovate other versions and delete all old CSD. Make sure the parcel and the CSD jar version matches.

 

 

Re: Spark 2.3 Csd failed to load

Rising Star

Deleted already, i experimented the other version that may be its build specific issues.

here are the current logs from scm manager

ERROR 1183968426@scm-web-53:com.cloudera.csd.components.CsdLocalRepository: Failed to load CSD from /opt/cloudera/csd/SPARK2_ON_YARN-2.3.0.cloudera4.jar, Error: CSD bundle data directory or Jar file doesn't exist

 

I am really looking forward to you , if we can solve this issue together