Community Articles

Find and share helpful community-sourced technical articles.
Labels (2)
avatar

In order to use a non operative system default version of python with Zeppelin Pyspark interpreter you need to add the following configuration in the interpreter configuration page:

76497-screen-shot-2018-05-29-at-100214-am.png

Previous to this configuration you need to install the python version you like to use along side the operative system one in separate folder.

Important: Changing the default os python could have an impact on other services and applications running on the machine. Hence we don't recommend doing so.

Once you have installed the python version you wish to use set the above configuration pointing to the path of the python binary.

After adding the above, save and restart the interpreter. To test changes:

76498-screen-shot-2018-05-29-at-100156-am.png\

Important: The new python version installed needs to be present on zeppelin machine and all cluster worker nodes. Use same directory and make sure access permissions to the python dir is correctly set.

10,209 Views
Comments

@Felix Albani According to the Spark2 document, it only supports Python 3.4+. How can I create an environment within Zeppelin that has python 3.4+? thank you

@Victor I think 3.6 is fine as per https://issues.apache.org/jira/browse/SPARK-19019 they fixed 3.6 issues and this got merged to 2.1.1, 2.2 and is part of 2.3 already.

@Felix Alban

that's good to know! I think it is important to know how to create virtual environments as it will be important for separate projects. If you know how, can you please list the instructions to create a virtual envirnment for a project ? thank you