Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

Solved Go to solution

How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

Explorer

I now use the sandbox at Windows 10 VirtualBox.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

Super Guru

You can install anaconda but will need to set the correct path during job execution

PYSPARK_PYTHON=/opt/anaconda/bin/python spark-submit spark-job.py

View solution in original post

3 REPLIES 3
Highlighted

Re: How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

Super Guru

You can install anaconda but will need to set the correct path during job execution

PYSPARK_PYTHON=/opt/anaconda/bin/python spark-submit spark-job.py

View solution in original post

Highlighted

Re: How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

Explorer

thanks for replies.

could I do it like followings? export PYSPARK_PYTHON=/opt/anaconda/bin/python spark-submit spark-job.py

Highlighted

Re: How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

Expert Contributor

Need to set new Python location via env variable PYSPARK_PYTHON

Don't have an account?
Coming from Hortonworks? Activate your account here