Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Impala-shell encounters ImportError when pwd contains python file whose name conflict with module

avatar
Explorer

When we execute impala-shell command at a directory contains user defined python script, and the script name conflicts with python module imported by impala-shell, we will encounter an Import Error.

 

How to reproduce:

  1. create abc.py
    $ touch abc.py

     

  2. execute impala-shell:
$ impala-shell
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "/usr/lib/impala-shell/impala_shell.py", line 24, in <module>
import getpass
File "/usr/lib64/python2.6/getpass.py", line 18, in <module>
import os, sys, warnings
File "/usr/lib64/python2.6/os.py", line 398, in <module>
import UserDict
File "/usr/lib64/python2.6/UserDict.py", line 82, in <module>
import _abcoll
File "/usr/lib64/python2.6/_abcoll.py", line 11, in <module>
from abc import ABCMeta, abstractmethod
ImportError: cannot import name ABCMeta

 

1 ACCEPTED SOLUTION

avatar
Champion
It is by design. The impala-shell scripts calls out that it will look in the current working directory.

Either workaround it or submit a request or submit your change back to the Apache Impala project.

View solution in original post

4 REPLIES 4

avatar
Champion
I am not a python expert here but I believe this is intended behavior as the import itself will look in the working directory prior to checking Python home, etc.

Yep confirmed in the Python docs. Well it is Python 3; should still be correct for Python 2.6 or 2.7.

https://docs.python.org/3/tutorial/modules.html#the-module-search-path

avatar
Champion
Better question: why are you putting python udfs in /usr/bin or /opt/cloudera/parcels/CDH/lib/impala-shell/?

avatar
Explorer

Not putting python script under /usr/bin, but my working directory.

 

For example, If someone else I don't known who write a python script under home directory:

[~]$ cat abc.py
#!/usr/bin/python
print "test script"

Now I cannot execute impala-shell anymore due to Import Error:

[~]$ impala-shell
test script
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "/usr/lib/impala-shell/impala_shell.py", line 24, in <module>
    import getpass
  File "/usr/lib64/python2.6/getpass.py", line 18, in <module>
    import os, sys, warnings
  File "/usr/lib64/python2.6/os.py", line 398, in <module>
    import UserDict
  File "/usr/lib64/python2.6/UserDict.py", line 82, in <module>
    import _abcoll
  File "/usr/lib64/python2.6/_abcoll.py", line 11, in <module>
    from abc import ABCMeta, abstractmethod
ImportError: cannot import name ABCMeta

I think impala-shell shall not include current directory in PYTHONPATH to avoid such error. 

 

avatar
Champion
It is by design. The impala-shell scripts calls out that it will look in the current working directory.

Either workaround it or submit a request or submit your change back to the Apache Impala project.