Reply
Expert Contributor
Posts: 71
Registered: ‎03-04-2015
Accepted Solution

What to do with Spark 2.0 CSD jar

[ Edited ]

The CSD download link given actually points to a jar file with the following directory tree:

 

$ jar tvf SPARK2_ON_YARN-2.0.0.cloudera.beta1.jar
0 Wed Sep 21 17:24:28 CDT 2016 descriptor/
0 Wed Sep 21 17:24:28 CDT 2016 scripts/
0 Wed Sep 21 17:24:28 CDT 2016 aux/
0 Wed Sep 21 17:24:28 CDT 2016 aux/client/
3312 Wed Sep 21 17:24:28 CDT 2016 images/icon.png
1711 Wed Sep 21 17:24:28 CDT 2016 aux/client/spark-env.sh
0 Wed Sep 21 17:24:28 CDT 2016 images/
18456 Wed Sep 21 17:24:28 CDT 2016 descriptor/service.sdl
0 Wed Sep 21 17:50:46 CDT 2016 meta/
20 Wed Sep 21 17:50:46 CDT 2016 meta/version
0 Wed Sep 21 17:50:58 CDT 2016 META-INF/
1813 Wed Sep 21 17:24:28 CDT 2016 scripts/control.sh
12362 Wed Sep 21 17:24:28 CDT 2016 scripts/common.sh
104 Wed Sep 21 17:50:58 CDT 2016 META-INF/MANIFEST.MF

 

 

Now, when the documentation specifies "install Spark2 CSD", which file(s) is it referring to exactly?  Just descriptor/service.sdl, or the entire jar to /opt/cloudera/csd?  The two scripts above look like operational scripts for CM.

 

Thanks,

Miles Yao

 

 

Posts: 62
Topics: 0
Kudos: 14
Blog Posts: 0
Ideas: 0
Solutions: 6
Registered: ‎11-26-2015

Re: What to do with Spark 2.0 CSD jar

Hello,

 

You have to move the SPARK2_ON_YARN-2.0.0.cloudera.beta1.jar to here:

 

/opt/cloudera/csd

 

and restart the Cloudera Manager with the following command:

 

service cloudera-scm-server restart

 

 

Here are the other steps:

 

https://www.cloudera.com/documentation/betas/spark2/latest/topics/spark2_installing.html

 

Best regards,

 

      Gabor

Expert Contributor
Posts: 71
Registered: ‎03-04-2015

Re: What to do with Spark 2.0 CSD jar

Yes, that works. 

"CSD file" sounds like a text config file.  Adding a simple description that it's a JAR in the instruction page would have clarified.

 

Thanks again.

Miles