Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Drill in Nifi (DBCPConnectionPool 1.5.0) error failed to process session due to java.lang.RuntimeException

avatar
Contributor

77676-drill-1.png

Hi,

I am trying to connect Apache Drill 1.13.0 from Apache NiFi 1.5.0 by DBCPConnectionPool 1.5.0

but I got error

06:17:54 UTC

ERROR

eec752d9-0163-1000-0000-0000293632ce

%%%%%

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] failed to process session due to java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class ${package.namespace.prefix}null not found: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class ${package.namespace.prefix}null not found

06:17:54 UTC

WARNING

eec752d9-0163-1000-0000-0000293632ce

%%%%

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] Processor Administratively Yielded for 1 sec due to processing failure

Thanks in advance,

Tom

10 REPLIES 10

avatar

Hi @Tommy

In the screenshot, I see you set /var/lib/dlfs/lib directory at 'Database Driver Location(s)'. How about setting the jar file path instead?

avatar
Contributor

Yes but when I set the jar file path instead, I get another error ("database-driver-locations ...")

77764-nifi-2.png

avatar
@Tommy

Please make sure that the jar file path is correct. If you are using a multi-node NiFi cluster, each node has to have the jar file at the specified path.

avatar
Contributor

@kkawamura,

I check that each node has to have the jar file under the path from Controller Services. Please see below.

77771-nifi-1-node2.png

77772-nifi-2-node3.png

77770-nifi-0-node1.png

77773-nifi-controller-servies.png

77774-nifi-controller-servies-error.png

avatar

@Tommy

The error message has been changed slightly, it seems the jar file is used now but failed with different reason. Would you please share the detail of the error? I think nifi-app.log has a stack-trace for the exception, which may be different than the one you posted before.

avatar
Contributor

07:24:56 UTC

WARNING

eec752d9-0163-1000-0000-0000293632ce

pas01nod02.internsg.net:9093

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] Processor Administratively Yielded for 1 sec due to processing failure

07:24:56 UTC

ERROR

eec752d9-0163-1000-0000-0000293632ce

pas01nod03.internsg.net:9093

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] failed to process session due to java.lang.IllegalStateException: Cannot invoke method public abstract java.sql.Connection org.apache.nifi.dbcp.DBCPService.getConnection() throws org.apache.nifi.processor.exception.ProcessException on Controller Service with identifier 0723417c-1d9b-1340-0000-00005b696e28 because the Controller Service is disabled: Cannot invoke method public abstract java.sql.Connection org.apache.nifi.dbcp.DBCPService.getConnection() throws org.apache.nifi.processor.exception.ProcessException on Controller Service with identifier 0723417c-1d9b-1340-0000-00005b696e28 because the Controller Service is disabled

07:24:56 UTC

WARNING

eec752d9-0163-1000-0000-0000293632ce

pas01nod03.internsg.net:9093

ExecuteSQL[id=eec752d9-0163-1000-0000-0000293632ce] Processor Administratively Yielded for 1 sec due to processing failure

07:25:05 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod01.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:05 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod02.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:05 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod03.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:35 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod03.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

07:25:35 UTC

ERROR

0723417c-1d9b-1340-0000-00005b696e28

pas01nod03.internsg.net:9093

DBCPConnectionPool[id=0723417c-1d9b-1340-0000-00005b696e28] Failed to invoke @OnEnabled method due to org.apache.nifi.reporting.InitializationException: Can't load Database Driver: Can't load Database Driver

logs-nifi.txt

avatar

I don't have a Drill environment to test with, but downloaded Drill JDBC driver and tested if I can load its Driver class with NiFi DBCPConnectionPool. Since drill driver needs other dependency jars, I needed to use the jar which contains all dependencies.

Please try drill-jdbc-all-1.13.0.jar instead.

apache-drill-1.13.0/jars/jdbc-driver/drill-jdbc-all-1.13.0.jar

avatar
Contributor

If I understood you correctly, I download Drill 1.13.0 (https://drill.apache.org/download/) and I copied the driver from apache-drill-1.13.0/jars/jdbc-driver/drill-jdbc-all-1.13.0.jar to /var/lib/dlfs/lib/drill-jdbc-all-1.13.0.jar, where access has all nodes with NiFi,

but when I started processor , I got the following error

77779-nifi-bulletin-board.png

nifi-logs.txt

avatar

I don't have much experience with Apache Drill, but looking at the source code, it looks like it fails at instantiating the class specified with 'hadoop.security.group.mapping' parameter. The parameter is usually defined at a Hadoop configuration file, 'core-default.xml'.

Are you aware of any configuration file specifying 'hadoop.security.group.mapping' parameter potentially being null?