question Re: ImportError: ruamel.yaml.comments while trying to run pyspark virtual env example with conda in Archives of Support Questions (Read Only)

ImportError: ruamel.yaml.comments while trying to run pyspark virtual env example with conda

kbadani — Thu, 09 Feb 2017 09:20:20 GMT

I’m trying to run a pyspark virtual env example with conda.

However, the application is failing with below error. Any pointers on this issue?

Traceback (most recent call last):

File "/usr/bin/conda", line 11, in <module>

load_entry_point('conda==4.2.7', 'console_scripts', 'conda')()

File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 561, in load_entry_point

return get_distribution(dist).load_entry_point(group, name)

File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2631, in load_entry_point

return ep.load()

File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2291, in load

return self.resolve()

File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2297, in resolve

module = __import__(self.module_name, fromlist=['__name__'], level=0)

File "/usr/lib/python2.7/site-packages/conda/cli/__init__.py", line 8, in <module>

from .main import main # NOQA

File "/usr/lib/python2.7/site-packages/conda/cli/main.py", line 46, in <module>

from ..base.context import context

File "/usr/lib/python2.7/site-packages/conda/base/context.py", line 18, in <module>

from ..common.configuration import (Configuration, MapParameter, PrimitiveParameter,

File "/usr/lib/python2.7/site-packages/conda/common/configuration.py", line 40, in <module>

from ruamel.yaml.comments import CommentedSeq, CommentedMap # pragma: no cover

ImportError: No module named ruamel.yaml.comments

Re: ImportError: ruamel.yaml.comments while trying to run pyspark virtual env example with conda

yvora — Thu, 09 Feb 2017 09:23:49 GMT

It looks like conda is not installed properly on the machine. You can use Anaconda 4.3.0 package to install conda properly on the machine.

Follow below steps to install conda.

Follow the steps mentioned on https://www.continuum.io/downloads#linux to download and install anaconda . ( Use anaconda2 with python2 and anaconda3 with python3)
Make sure the conda installed by above package is a default one on all the hosts. You can update symlink for /usr/bin/conda to /root/anaconda2/bin/conda (ln -s /root/anaconda2/bin/conda /usr/bin/conda)

Re: ImportError: ruamel.yaml.comments while trying to run pyspark virtual env example with conda

agauthier — Thu, 09 Feb 2017 09:59:29 GMT

Assuming you're using HDP 2.5 sandbox, another option would be to deploy Zeppelin service in Amabari. The modules above are also included in Zeppelin.

Re: ImportError: ruamel.yaml.comments while trying to run pyspark virtual env example with conda

kbadani — Thu, 09 Feb 2017 10:04:18 GMT

@yvora Thanks. Looks like that was an issue