Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How Install and set anaconda instead of built-in python at HDP2.5 sandbox for Spark MLlib?

avatar
Explorer

I now use the sandbox at Windows 10 VirtualBox.

1 ACCEPTED SOLUTION

avatar
Master Guru

You can install anaconda but will need to set the correct path during job execution

PYSPARK_PYTHON=/opt/anaconda/bin/python spark-submit spark-job.py

View solution in original post

3 REPLIES 3

avatar
Master Guru

You can install anaconda but will need to set the correct path during job execution

PYSPARK_PYTHON=/opt/anaconda/bin/python spark-submit spark-job.py

avatar
Explorer

thanks for replies.

could I do it like followings? export PYSPARK_PYTHON=/opt/anaconda/bin/python spark-submit spark-job.py

avatar
Super Collaborator

Need to set new Python location via env variable PYSPARK_PYTHON