Created 02-17-2016 12:11 PM
Hi all,
I just upgrade spark 1.5 to 1.6 on HDP 2.3
I'm following this tutorial here
and i got this error while try to run pysprak from terminal
/usr/bin/pyspark: line 22: /usr/bin/load-spark-env.sh: No such file or directory /usr/bin/spark-class: line 23: /usr/bin/load-spark-env.sh: No such file or directory ls: cannot access /usr/assembly/target/scala-: No such file or directory Failed to find Spark assembly in /usr/assembly/target/scala-. You need to build Spark before running this program.
Created 02-17-2016 12:36 PM
@cokorda putra susila make sure to install Spark client on the host you are running this from. Go to host components page and install Spark client.
Created 02-17-2016 02:10 PM
Great, @cokorda putra susila please accept one of the answers as best practice.
Created 02-17-2016 06:53 PM
Officially upgrade from Spark 1.5.2 > 1.6 is part of HDP 2.4 that is coming out March 1st.
The Spark 1.6 tutorial is for evaluating Spark 1.6 and not for evaluation of upgrade instructions.
Created 02-17-2016 07:05 PM