Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Impala-shell encounters ImportError when pwd contains python file whose name conflict with module

Solved Go to solution
Highlighted

Impala-shell encounters ImportError when pwd contains python file whose name conflict with module

Explorer

When we execute impala-shell command at a directory contains user defined python script, and the script name conflicts with python module imported by impala-shell, we will encounter an Import Error.

 

How to reproduce:

  1. create abc.py
    $ touch abc.py

     

  2. execute impala-shell:
$ impala-shell
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "/usr/lib/impala-shell/impala_shell.py", line 24, in <module>
import getpass
File "/usr/lib64/python2.6/getpass.py", line 18, in <module>
import os, sys, warnings
File "/usr/lib64/python2.6/os.py", line 398, in <module>
import UserDict
File "/usr/lib64/python2.6/UserDict.py", line 82, in <module>
import _abcoll
File "/usr/lib64/python2.6/_abcoll.py", line 11, in <module>
from abc import ABCMeta, abstractmethod
ImportError: cannot import name ABCMeta

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

Champion
It is by design. The impala-shell scripts calls out that it will look in the current working directory.

Either workaround it or submit a request or submit your change back to the Apache Impala project.
4 REPLIES 4

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

Champion
I am not a python expert here but I believe this is intended behavior as the import itself will look in the working directory prior to checking Python home, etc.

Yep confirmed in the Python docs. Well it is Python 3; should still be correct for Python 2.6 or 2.7.

https://docs.python.org/3/tutorial/modules.html#the-module-search-path

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

Champion
Better question: why are you putting python udfs in /usr/bin or /opt/cloudera/parcels/CDH/lib/impala-shell/?

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

Explorer

Not putting python script under /usr/bin, but my working directory.

 

For example, If someone else I don't known who write a python script under home directory:

[~]$ cat abc.py
#!/usr/bin/python
print "test script"

Now I cannot execute impala-shell anymore due to Import Error:

[~]$ impala-shell
test script
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "/usr/lib/impala-shell/impala_shell.py", line 24, in <module>
    import getpass
  File "/usr/lib64/python2.6/getpass.py", line 18, in <module>
    import os, sys, warnings
  File "/usr/lib64/python2.6/os.py", line 398, in <module>
    import UserDict
  File "/usr/lib64/python2.6/UserDict.py", line 82, in <module>
    import _abcoll
  File "/usr/lib64/python2.6/_abcoll.py", line 11, in <module>
    from abc import ABCMeta, abstractmethod
ImportError: cannot import name ABCMeta

I think impala-shell shall not include current directory in PYTHONPATH to avoid such error. 

 

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

Champion
It is by design. The impala-shell scripts calls out that it will look in the current working directory.

Either workaround it or submit a request or submit your change back to the Apache Impala project.
Don't have an account?
Coming from Hortonworks? Activate your account here