Created on 05-30-2018 01:27 PM - edited 08-17-2019 07:16 AM
In order to use a non operative system default version of python with Zeppelin Pyspark interpreter you need to add the following configuration in the interpreter configuration page:
Previous to this configuration you need to install the python version you like to use along side the operative system one in separate folder.
Important: Changing the default os python could have an impact on other services and applications running on the machine. Hence we don't recommend doing so.
Once you have installed the python version you wish to use set the above configuration pointing to the path of the python binary.
After adding the above, save and restart the interpreter. To test changes:
\
Important: The new python version installed needs to be present on zeppelin machine and all cluster worker nodes. Use same directory and make sure access permissions to the python dir is correctly set.
Created on 05-31-2018 01:07 PM
@Felix Albani According to the Spark2 document, it only supports Python 3.4+. How can I create an environment within Zeppelin that has python 3.4+? thank you
Created on 05-31-2018 01:55 PM
@Victor I think 3.6 is fine as per https://issues.apache.org/jira/browse/SPARK-19019 they fixed 3.6 issues and this got merged to 2.1.1, 2.2 and is part of 2.3 already.
Created on 05-31-2018 02:00 PM
that's good to know! I think it is important to know how to create virtual environments as it will be important for separate projects. If you know how, can you please list the instructions to create a virtual envirnment for a project ? thank you