Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]
Created 02-17-2016 12:11 PM
Hi all,
I just upgrade spark 1.5 to 1.6 on HDP 2.3
I'm following this tutorial here
and i got this error while try to run pysprak from terminal
/usr/bin/pyspark: line 22: /usr/bin/load-spark-env.sh: No such file or directory /usr/bin/spark-class: line 23: /usr/bin/load-spark-env.sh: No such file or directory ls: cannot access /usr/assembly/target/scala-: No such file or directory Failed to find Spark assembly in /usr/assembly/target/scala-. You need to build Spark before running this program.
Created 02-17-2016 12:36 PM
@cokorda putra susila make sure to install Spark client on the host you are running this from. Go to host components page and install Spark client.
Created 02-17-2016 02:10 PM
Great, @cokorda putra susila please accept one of the answers as best practice.
Created 02-17-2016 06:53 PM
Officially upgrade from Spark 1.5.2 > 1.6 is part of HDP 2.4 that is coming out March 1st.
The Spark 1.6 tutorial is for evaluating Spark 1.6 and not for evaluation of upgrade instructions.
Created 02-17-2016 07:05 PM