Drill in Nifi (DBCPConnectionPool 1.5.0) error failed to process session due to java.lang.RuntimeException

[#1]

Hi,

I am trying to connect Apache Drill 1.13.0 from Apache NiFi 1.5.0 by DBCPConnectionPool 1.5.0

but I got error

06:17:54 UTC

ERROR

eec752d9-0163-1000-0000-0000293632ce

%%%%%

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] failed to process session due to java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class ${package.namespace.prefix}null not found: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class ${package.namespace.prefix}null not found

06:17:54 UTC

WARNING

eec752d9-0163-1000-0000-0000293632ce

%%%%

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] Processor Administratively Yielded for 1 sec due to processing failure

Thanks in advance,

Tom

[#2]

Hi @Tommy

In the screenshot, I see you set /var/lib/dlfs/lib directory at 'Database Driver Location(s)'. How about setting the jar file path instead?

[#3]

Yes but when I set the jar file path instead, I get another error ("database-driver-locations ...")

[#4]

@Tommy

Please make sure that the jar file path is correct. If you are using a multi-node NiFi cluster, each node has to have the jar file at the specified path.

[#5]

@kkawamura,

I check that each node has to have the jar file under the path from Controller Services. Please see below.

[#6]

@Tommy

The error message has been changed slightly, it seems the jar file is used now but failed with different reason. Would you please share the detail of the error? I think nifi-app.log has a stack-trace for the exception, which may be different than the one you posted before.

[#7]

07:24:56 UTC

WARNING

eec752d9-0163-1000-0000-0000293632ce

pas01nod02.internsg.net:9093

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] Processor Administratively Yielded for 1 sec due to processing failure

07:24:56 UTC

ERROR

eec752d9-0163-1000-0000-0000293632ce

pas01nod03.internsg.net:9093

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] failed to process session due to java.lang.IllegalStateException: Cannot invoke method public abstract java.sql.Connection org.apache.nifi.dbcp.DBCPService.getConnection() throws org.apache.nifi.processor.exception.ProcessException on Controller Service with identifier 0723417c-1d9b-1340-0000-00005b696e28 because the Controller Service is disabled: Cannot invoke method public abstract java.sql.Connection org.apache.nifi.dbcp.DBCPService.getConnection() throws org.apache.nifi.processor.exception.ProcessException on Controller Service with identifier 0723417c-1d9b-1340-0000-00005b696e28 because the Controller Service is disabled

07:24:56 UTC

WARNING

eec752d9-0163-1000-0000-0000293632ce

pas01nod03.internsg.net:9093

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] Processor Administratively Yielded for 1 sec due to processing failure

07:25:05 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod01.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:05 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod02.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:05 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod03.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:35 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod03.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:35 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod03.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

logs-nifi.txt

[#8]

I don't have a Drill environment to test with, but downloaded Drill JDBC driver and tested if I can load its Driver class with NiFi DBCPConnectionPool. Since drill driver needs other dependency jars, I needed to use the jar which contains all dependencies.

Please try drill-jdbc-all-1.13.0.jar instead.

apache-drill-1.13.0/jars/jdbc-driver/drill-jdbc-all-1.13.0.jar

[#9]

If I understood you correctly, I download Drill 1.13.0 (https://drill.apache.org/download/) and I copied the driver from apache-drill-1.13.0/jars/jdbc-driver/drill-jdbc-all-1.13.0.jar to /var/lib/dlfs/lib/drill-jdbc-all-1.13.0.jar, where access has all nodes with NiFi,

but when I started processor , I got the following error

nifi-logs.txt

[#10]

I don't have much experience with Apache Drill, but looking at the source code, it looks like it fails at instantiating the class specified with 'hadoop.security.group.mapping' parameter. The parameter is usually defined at a Hadoop configuration file, 'core-default.xml'.

Are you aware of any configuration file specifying 'hadoop.security.group.mapping' parameter potentially being null?