Configure Zeppelin Spark interpreter to work with python3

[#1]

I'm using HDP-3.1 (spark2.3, python 3.6, Centos7)

I was able to configure it to work with python2.7, but with 3.6 I got the following error when running a notebook:

Exception: Python in worker has different version 2.7 than that in driver 3.6, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.

I have python 3.6 installed on all of the workers and environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set (see my configuration bellow):

Am I missing something?

[#2]

@fklezin 

I think your zeppelin is not aware of the python version in the Zeppelin config at /usr/hdp/current/zeppelin-server/conf/interpreter.json, and change the below line 30 in the config:

"zeppelin.pyspark.python": {
"type": "string",
"name": "zeppelin.pyspark.python",
"value": "python"
},

To
"zeppelin.pyspark.python": {
"type": "string",
"name": "zeppelin.pyspark.python3",
"value": "python"
},

Make sure you have these values in Ambari UI--> Zeppelin-->Config-->Advanced zeppelin-env

export PYSPARK_PYTHON=python3
export PYSPARK_DRIVER_PYTHON=python3

Restart Zeppelin and retry

[#3]

@fklezin 

Any updates do you still need help?