question Re: Where is $SPARK_HOME ? in Archives of Support Questions (Read Only)

Where is $SPARK_HOME ?

bernhard_bb_sch — Wed, 14 Sep 2016 18:18:03 GMT

I want to execute "./sbin/start-master.sh", but I'm unable to locate this file in the /sbin folder. I wanted to find the file using echo $SPARK_HOME but the result is empty. I'm on HDP 2.4.

That leads to the question: Where is Spark?

Re: Where is $SPARK_HOME ?

mjohansson — Wed, 14 Sep 2016 18:26:05 GMT

Normally it's under /usr/hdp/current/spark-client

Re: Where is $SPARK_HOME ?

bernhard_bb_sch — Wed, 14 Sep 2016 18:26:36 GMT

Sometimes asking the question is half the answer:

find . -name start-master.sh

gave

./usr/hdp/2.4.0.0-169/spark/sbin/start-master.sh

Re: Where is $SPARK_HOME ?

pvillard — Wed, 14 Sep 2016 18:29:34 GMT

You need to go where your Spark client is installed. Depending of your install/OS, it may be :

/usr/hdp/current/spark-client/sbin

Hope this helps.

Re: Where is $SPARK_HOME ?

mjohansson — Wed, 14 Sep 2016 18:35:09 GMT

Sorry, too quick on the trigger.

Normally SPARK_HOME=/usr/hdp/current/spark-client

In my (old) sandbox the start_master.sh script is located at /usr/hdp/2.4.0.0-169/spark/sbin/start-master.sh.