Impala.dbapi error

drgenious — Mon, 21 Feb 2022 08:22:19 GMT

I am trying to run a script in oozie and every time I receive the below error regarding impala.dbapi. The module is inserted correctly in the script.

Stdoutput Traceback (most recent call last):
Stdoutput File "/tmp/sorting_table.py", line 8, in <module>
Stdoutput from impala.dbapi import connect
Stdoutput ImportError: No module named impala.dbapi
Exit code of the Shell command 1
<<< Invocation of Shell command completed <<<

java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
at org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141)
Caused by: org.apache.oozie.action.hadoop.LauncherMainException
at org.apache.oozie.action.hadoop.ShellMain.run(ShellMain.java:76)
at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104)
at org.apache.oozie.action.hadoop.ShellMain.main(ShellMain.java:63)

Script import libraries:

from pyspark import SparkContext
from pyspark.sql import SparkSession
from datetime import datetime,timedelta
import ssl
from impala.dbapi import connect
import thrift_sasl
import os

Re: Impala.dbapi error

jAnshula — Wed, 03 Apr 2024 09:08:04 GMT

Hi @drgenious

First, please test your script outside of Oozie.

If it is working outside of Oozie, then it should work from Oozie as well.

As for error "No module named impala.dbapi" it could be that there is some version dependency issue with impyla and its related libraries

refer ---> https://github.com/cloudera/impyla/issues/227

question Re: Impala.dbapi error in Support Questions

Impala.dbapi error

Re: Impala.dbapi error