Reply
Explorer
Posts: 13
Registered: ‎05-19-2018

JDBC Driver Upgrade to 2.6.9 fails on Apache NiFi

[ Edited ]

We plan to switch on impala server side to the newest CDH 6.2.0 release, hence we would like to upgrade as well the Cloudera JDBC Connector from 2.6.4 to 2.6.9 (https://www.cloudera.com/documentation/other/connectors/impala-jdbc/Cloudera-JDBC-Driver-for-Impala-...) . However as soon as we switch “ImpalaJDBC41.jar” to the new 2.6.9 version within Apache NiFi 1.9.2, we get the following stacktrace from NiFi if we start an ExecuteSQL processor.

2019-04-26 18:30:11,497 ERROR [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.ExecuteSQL ExecuteSQL[id=5957fa4c-016a-1000-ffff-ffffcdfcd7bc] ExecuteSQL[id=5957fa4c-016a-1000-ffff-ffffcdfcd7bc] failed to process session due to java.lang.ExceptionInInitializerError; Processor Administratively Yielded for 1 sec: java.lang.ExceptionInInitializerError
java.lang.ExceptionInInitializerError: null
        at com.cloudera.impala.jdbc41.internal.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)
        at com.cloudera.impala.jdbc41.internal.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)
        at com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
        at com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
        at com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
        at com.cloudera.impala.jdbc41.internal.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
        at com.cloudera.impala.jdbc41.internal.apache.thrift.transport.TIOStreamTransport.<clinit>(TIOStreamTransport.java:38)
        at com.cloudera.impala.hivecommon.api.TETSSLTransportFactory.createClient(Unknown Source)
        at com.cloudera.impala.hivecommon.api.TETSSLTransportFactory.getClientSocket(Unknown Source)
        at com.cloudera.impala.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
        at com.cloudera.impala.hivecommon.api.ServiceDiscoveryFactory.createClient(Unknown Source)
        at com.cloudera.impala.hivecommon.core.HiveJDBCCommonConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.impala.core.ImpalaJDBCConnection.establishConnection(Unknown Source)
        at com.cloudera.impala.jdbc.core.LoginTimeoutConnection.connect(Unknown Source)
        at com.cloudera.impala.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
        at com.cloudera.impala.jdbc.common.AbstractDriver.connect(Unknown Source)
        at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:53)
        at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:291)
        at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2395)
        at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2381)
        at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2110)
        at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1563)
        at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:470)
        at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:49)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
        at com.sun.proxy.$Proxy133.getConnection(Unknown Source)
        at org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:222)
        at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
        at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
        at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
        at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND bound slf4j-log4j12.jar on the class path, preempting StackOverflowError. See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
        at com.cloudera.impala.jdbc41.internal.slf4j.impl.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:54)
        ... 43 common frames omitted

 

For me this looks like a logging library log4j incompatibility between NiFi 1.9.2 and the impala JDBC driver starting from release 2.6.6. The driver works in other tools like Zeppelin or DBVisualizer.

 

Would be great to see that working again. Any ideas how we get a fix or how we can fix this issue?

 

Cheers Josef

Posts: 1,903
Kudos: 435
Solutions: 307
Registered: ‎07-31-2013

Re: JDBC Driver Upgrade to 2.6.9 fails on Apache NiFi

The primary change appears to be that Impala JDBC 2.6 drivers began to shade-in slf4j jars and other dependencies instead of offering them as separate jars as in 2.5.

Do you perhaps have older slf4j-log4j12*.jar and slf4j-api-*.jar still leftover when you upgraded the driver jar? Try removing them away to move past this, as the new driver jar does not require them to be independently present.

Apache NiFi uses the log4j-over-slf4j*.jar so that likely shouldn't be removed.
Explorer
Posts: 13
Registered: ‎05-19-2018

Re: JDBC Driver Upgrade to 2.6.9 fails on Apache NiFi

There are no leftovers from the old version and as you mentioned the log4j JARS from NiFi can't be removed. So we are stuck here and I guess it will not get any better with newer driver versions :-S...