question Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul in Archives of Support Questions (Read Only)

Impala-shell encounters ImportError when pwd contains python file whose name conflict with module

linhao — Fri, 16 Sep 2022 11:43:17 GMT

When we execute impala-shell command at a directory contains user defined python script, and the script name conflicts with python module imported by impala-shell, we will encounter an Import Error.

How to reproduce:

create abc.py
```
$ touch abc.py
```
execute impala-shell:

$ impala-shell
'import site' failed; use -v for traceback
Traceback (most recent call last):
File "/usr/lib/impala-shell/impala_shell.py", line 24, in <module>
import getpass
File "/usr/lib64/python2.6/getpass.py", line 18, in <module>
import os, sys, warnings
File "/usr/lib64/python2.6/os.py", line 398, in <module>
import UserDict
File "/usr/lib64/python2.6/UserDict.py", line 82, in <module>
import _abcoll
File "/usr/lib64/python2.6/_abcoll.py", line 11, in <module>
from abc import ABCMeta, abstractmethod
ImportError: cannot import name ABCMeta

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

mbigelow — Fri, 09 Jun 2017 02:43:38 GMT

I am not a python expert here but I believe this is intended behavior as the import itself will look in the working directory prior to checking Python home, etc.

Yep confirmed in the Python docs. Well it is Python 3; should still be correct for Python 2.6 or 2.7.

https://docs.python.org/3/tutorial/modules.html#the-module-search-path

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

mbigelow — Fri, 09 Jun 2017 02:44:17 GMT

Better question: why are you putting python udfs in /usr/bin or /opt/cloudera/parcels/CDH/lib/impala-shell/?

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

linhao — Fri, 09 Jun 2017 03:57:17 GMT

Not putting python script under /usr/bin, but my working directory.

For example, If someone else I don't known who write a python script under home directory:

[~]$ cat abc.py
#!/usr/bin/python
print "test script"

Now I cannot execute impala-shell anymore due to Import Error:

[~]$ impala-shell
test script
'import site' failed; use -v for traceback
Traceback (most recent call last):
  File "/usr/lib/impala-shell/impala_shell.py", line 24, in <module>
    import getpass
  File "/usr/lib64/python2.6/getpass.py", line 18, in <module>
    import os, sys, warnings
  File "/usr/lib64/python2.6/os.py", line 398, in <module>
    import UserDict
  File "/usr/lib64/python2.6/UserDict.py", line 82, in <module>
    import _abcoll
  File "/usr/lib64/python2.6/_abcoll.py", line 11, in <module>
    from abc import ABCMeta, abstractmethod
ImportError: cannot import name ABCMeta

I think impala-shell shall not include current directory in PYTHONPATH to avoid such error.

Re: Impala-shell encounters ImportError when pwd contains python file whose name conflict with modul

mbigelow — Mon, 12 Jun 2017 17:20:59 GMT

It is by design. The impala-shell scripts calls out that it will look in the current working directory.

Either workaround it or submit a request or submit your change back to the Apache Impala project.