Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Concerns with installing Python 3.X on Hadoop Cluster

Solved Go to solution

Concerns with installing Python 3.X on Hadoop Cluster

Hi All,

I am aware that HDP and Ambari need Python 2.X to run it's services. However are there any concerns with installing Python 3.5.1 to be used for data processing?

Thanks,

Andrew

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Concerns with installing Python 3.X on Hadoop Cluster

Hi @Andrew Watson

As long as it's installed out of the default system paths, and only called by whatever processing scripts or tools you're thinking of, I don't see that being an issue at all.

It'll require careful installation and ongoing care though, just from a security and patching perspective.

View solution in original post

5 REPLIES 5
Highlighted

Re: Concerns with installing Python 3.X on Hadoop Cluster

Hi @Andrew Watson

As long as it's installed out of the default system paths, and only called by whatever processing scripts or tools you're thinking of, I don't see that being an issue at all.

It'll require careful installation and ongoing care though, just from a security and patching perspective.

View solution in original post

Highlighted

Re: Concerns with installing Python 3.X on Hadoop Cluster

Guru

@Andrew Watson

If you are also looking at mutiple python library versions, take a look at virtualenv. This makes managing multiple python environments easier.

Highlighted

Re: Concerns with installing Python 3.X on Hadoop Cluster

Contributor

It is also worth while checking pyenv: https://github.com/yyuu/pyenv

From their home page:

pyenv does...

  • Let you change the global Python version on a per-user basis.
  • Provide support for per-project Python versions.
  • Allow you to override the Python version with an environment variable.
  • Search commands from multiple versions of Python at a time. This may be helpful to test across Python versions with tox.
Highlighted

Re: Concerns with installing Python 3.X on Hadoop Cluster

Explorer

hello andrew,

have you installed the python 3.x version? if so how is it running?

i am also trying to install python 3.x in my HDP. can you guide me in this?

Highlighted

Re: Concerns with installing Python 3.X on Hadoop Cluster

Expert Contributor

@Andrew Watson @Dave Russell

Have you tried to install python 3 on hdp? what did you use to install? virtualenv? conda env? which works better? do you have some instructions? is this advisable to do on production environment?

Don't have an account?
Coming from Hortonworks? Activate your account here