zeppelin pyspark cannot run with different minor version

wazzu62 — Wed, 10 May 2017 00:38:43 GMT

HDP 2.5.3 Spark 1.6.x.2.5 Zeppelin Notebook 0.6.0.2.5

Just installed zeppelin, trying to run some of the examples. On the zeppelin notebook server I have python 2.7 installed (/usr/bin/python). On the other nodes in the cluster I have python 2.6 (/usr/bin/python) and 2.7 (/usr/local/bin/python) installed.

I am runnning the "Hello World" example in zeppelin. Most steps run, but some fail with this error

Exception: Python in worker has different version 2.6 than that in driver 2.7, PySpark cannot run with different minor versions

In the stack trace I can see the node that I assume is the worker. It is not a spark client but I created /etc/spark/conf/spark-env.sh with the following

export PYSPARK_PYTHON=/usr/local/bin/python

export PYSPARK_DRIVER_PYTHON=python

pointing to python 2.7, the same as the version that is running on the Zeppelin notebook.

I am still getting the error. I am leery about removing the python 2.6 on the other nodes but am not sure how to get around this.

Re: zeppelin pyspark cannot run with different minor version

kbadani — Thu, 11 May 2017 23:56:55 GMT

@Jon Page

Can you please refer to my answere here:

https://community.hortonworks.com/questions/92000/error-running-zeppelin-pyspark-interpreter-with-py.html#answer-92029

question Re: zeppelin pyspark cannot run with different minor version in Archives of Support Questions (Read Only)

zeppelin pyspark cannot run with different minor version

Re: zeppelin pyspark cannot run with different minor version