HDP 2.5.3 Spark 1.6.x.2.5 Zeppelin Notebook 0.6.0.2.5
Just installed zeppelin, trying to run some of the examples. On the zeppelin notebook server I have python 2.7 installed (/usr/bin/python). On the other nodes in the cluster I have python 2.6 (/usr/bin/python) and 2.7 (/usr/local/bin/python) installed.
I am runnning the "Hello World" example in zeppelin. Most steps run, but some fail with this error
Exception: Python in worker has different version 2.6 than that in driver 2.7, PySpark cannot run with different minor versions
In the stack trace I can see the node that I assume is the worker. It is not a spark client but I created /etc/spark/conf/spark-env.sh with the following
export PYSPARK_PYTHON=/usr/local/bin/python
export PYSPARK_DRIVER_PYTHON=python
pointing to python 2.7, the same as the version that is running on the Zeppelin notebook.
I am still getting the error. I am leery about removing the python 2.6 on the other nodes but am not sure how to get around this.
