ImportError: ruamel.yaml.comments while trying to run pyspark virtual env example with conda

[#1]

I’m trying to run a pyspark virtual env example with conda.

However, the application is failing with below error. Any pointers on this issue?

Traceback (most recent call last):

  File "/usr/bin/conda", line 11, in <module>

    load_entry_point('conda==4.2.7', 'console_scripts', 'conda')()

  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 561, in load_entry_point

    return get_distribution(dist).load_entry_point(group, name)

  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2631, in load_entry_point

    return ep.load()

  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2291, in load

    return self.resolve()

  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2297, in resolve

    module = __import__(self.module_name, fromlist=['__name__'], level=0)

  File "/usr/lib/python2.7/site-packages/conda/cli/__init__.py", line 8, in <module>

    from .main import main  # NOQA

  File "/usr/lib/python2.7/site-packages/conda/cli/main.py", line 46, in <module>

    from ..base.context import context

  File "/usr/lib/python2.7/site-packages/conda/base/context.py", line 18, in <module>

    from ..common.configuration import (Configuration, MapParameter, PrimitiveParameter,

  File "/usr/lib/python2.7/site-packages/conda/common/configuration.py", line 40, in <module>

    from ruamel.yaml.comments import CommentedSeq, CommentedMap  # pragma: no cover

ImportError: No module named ruamel.yaml.comments

[#2 · Accepted solution]

It looks like conda is not installed properly on the machine. You can use Anaconda 4.3.0 package to install conda properly on the machine.

Follow below steps to install conda.

• Follow the steps mentioned on https://www.continuum.io/downloads#linux to download and install anaconda . ( Use anaconda2 with python2 and anaconda3 with python3)
• Make sure the conda installed by above package is a default one on all the hosts. You can update symlink for /usr/bin/conda to /root/anaconda2/bin/conda (ln -s /root/anaconda2/bin/conda /usr/bin/conda)

[#3]

@yvora Thanks. Looks like that was an issue

[#4]

Assuming you're using HDP 2.5 sandbox, another option would be to deploy Zeppelin service in Amabari. The modules above are also included in Zeppelin.