Reply
Explorer
Posts: 6
Registered: ‎06-29-2017

Spark2 . Error deploying and status ok

[ Edited ]

I have installed Spark2 into Cloudera Manager, service is added in ok status although cloudera could not deploy well . Now I run 'spark2-shell' shows a error message:

 

[u_m1@cm bin]# ./spark2-shell
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream
at org.apache.spark.deploy.SparkSubmitArguments$$anonfun$mergeDefaultSparkProperties$1.apply(SparkSubmitArguments.scala:118)
at org.apache.spark.deploy.SparkSubmitArguments$$anonfun$mergeDefaultSparkProperties$1.apply(SparkSubmitArguments.scala:118)
at scala.Option.getOrElse(Option.scala:121)
at org.apache.spark.deploy.SparkSubmitArguments.mergeDefaultSparkProperties(SparkSubmitArguments.scala:118)
at org.apache.spark.deploy.SparkSubmitArguments.<init>(SparkSubmitArguments.scala:104)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:117)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)

 

 

How can i deploy correctly the service?. Some recomendation?

Cloudera Employee
Posts: 25
Registered: ‎11-16-2015

Re: Spark2 . Error deploying and status ok

This error is almost always a result of not having Spark2 gateway role configured on the host from where you're trying to run spark2-shell (CM > Spark2 > Instances > Gateway). I'd ensure that the steps to add Spark2 service including CSD are correctly followed including a restart CM and CMS and would double check that the client configuration is correctly deployed (CM > Cluster Name Drop Down menu> Deploy Client Configuration).

 

If all is well, you should see the alternatives pointing to the /etc/spark2/conf...  (required for running spark2-shell)

[u_m1@cm bin]# alternatives --display spark2-conf
spark2-conf - status is auto.
link currently points to /etc/spark2/conf.cloudera.spark2_on_yarn
/opt/cloudera/parcels/SPARK2-2.2.0.cloudera1-1.cdh5.12.0.p0.142354/etc/spark2/conf.dist - priority 10
/etc/spark2/conf.cloudera.spark2_on_yarn - priority 51
Current `best' version is /etc/spark2/conf.cloudera.spark2_on_yarn.

 

Since you mentioned "service is added in ok status although cloudera could not deploy well "

Can you share with us what was the error? Maybe you might want to remove the service from CM, re-add it by ensuring that service is configured according to the document? Let us know.

 

Explorer
Posts: 6
Registered: ‎06-29-2017

Re: Spark2 . Error deploying and status ok

I added again spark2 service (with gateway in all nodes), all was ok ( I haven´t seen any error ), but command 'spark2-shell' shows the same error.

 

alternatives of spark2:

 

[root@node-r3 ~]# alternatives --display spark2-conf
spark2-conf - status is auto.
link currently points to /opt/cloudera/parcels/SPARK2-2.0.0.cloudera2-1.cdh5.7.0.p0.118100/etc/spark2/conf.dist
/opt/cloudera/parcels/SPARK2-2.0.0.cloudera2-1.cdh5.7.0.p0.118100/etc/spark2/conf.dist - priority 10
Current `best' version is /opt/cloudera/parcels/SPARK2-2.0.0.cloudera2-1.cdh5.7.0.p0.118100/etc/spark2/conf.dist.

Cloudera Employee
Posts: 25
Registered: ‎11-16-2015

Re: Spark2 . Error deploying and status ok

[ Edited ]
link currently points to /opt/cloudera/parcels/SPARK2-2.0.0.cloudera2-1.cdh5.7.0.p0.118100/etc/spark2/conf.dist

Hmm, that is our problem. If the spark2 gateway instance and the client configurations are rightly deployed then the link would automatically point to /etc/spark2/conf.cloudera.spark2_on_yarn. Could you possibly share a print screen of the CM UI > Home page (just displaying all the services) and a print-screen of CM UI > Spark2_on_yarn> Instances ? 

 

Does redeploying client-configurations from CM UI > Cluster name (drop down icon) > Deploy Client Configuration shows all okay or is there an error?

 

From host 'node-r3' can you also share $ ls -l /etc/spark2/ to see if there exists a directory /etc/spark2/conf.cloudera.spark2_on_yarn

 

BTW, I believe you can just run spark2-shell command instead of going into the spark2 parcel bin directory and launching ./spark2-shell

Explorer
Posts: 6
Registered: ‎06-29-2017

Re: Spark2 . Error deploying and status ok

 

I response your questions:

 

Could you possibly share a print screen of the CM UI > Home page:

 

Captura2.JPG

 

CM UI > Spark2_on_yarn> Instances

 

Captura3.JPG

 

 CM UI > Cluster name (drop down icon) > Deploy Client Configuration shows all okay or is there an error?

 It was ok

 

 

ls -ltr /etc/spark2/
lrwxrwxrwx 1 root root 29 nov 21 04:07 conf -> /etc/alternatives/spark2-conf

 

spark2-shell shows the same error, wherever I launch that command. :(

 

 

Announcements