Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 11-07-2016 12:17 PM
I recently installed HDP 2.5 on a 3 node cluster using Ambari. I have installed Spark 2.0 along with it and have set SPARK_MAJOR_VERSION to 2. The spark-shell starts fine for 1.6.2 but throws the following error for Spark 2.0.
spark-shell SPARK_MAJOR_VERSION is set to 2, using Spark2 /usr/hdp/current/spark2-client/bin/spark-shell: line 57: /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/spark-2.0.0-bin-hadoop2.6/bin/spark-submit: No such file or directory
Can someone help with this error?
Created 11-07-2016 12:34 PM
Just realized SPARK_HOME wasn't set properly...SPARK_HOME was set to the old cloudera version we had on that cluster...resetting it made it work
Created 11-07-2016 12:30 PM
"/opt/cloudera/parcels/CDH-5.7..." doesn't sound like HDP 2.5. Do you have a conflicting install of CDH and HDP?
Created 11-07-2016 12:34 PM
Just realized SPARK_HOME wasn't set properly...SPARK_HOME was set to the old cloudera version we had on that cluster...resetting it made it work