Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

[#1]

Hi,

I have Nifi 2.0-M2 and Apache Ignite 2.16 database running on my localhost.

http://localhost:8443/nifi

jdbc:ignite:thin://localhost:10800  (I am able to connect to Ignite localhost db from Dbeaver)

I have copied ignite-core-2.16.0.jar file to nifi lib directory (before starting nifi) and tried to create DBCPConnectionPool controller service in Nifi to connect to Ignite database.  

DB Controller service returns the below error.

Failed to invoke @OnEnabled method: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource - Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-10"]

Can you let me know what am I missing here?  Please let me know if more details are required.

Screenshot of DB controller service

[#2]

Did you update ignite-core-2.16.0.jar file location path under the Database driver location? 

[#3]

@ckumar   Yes I've placed ignite-core-2.16.0.jar file in nifi lib directory. 

[#4]

The third property of the DBCPConnectionPool called "Database Driver Location" needs to be filled with the path to your nifi lib directory and then it should successfully load the JDBC driver class.

[#5]

@Jim_Steinebrey @shiva239 

I would avoid adding any additional custom jars or nars to the NiFi lib directory.  This will complicate future upgrade efforts.  Instead you should create a new directory outside NiFi's default directory tree for your custom jars that your specific dataflow(s) depend on. Make sure that the custom driver directory is both reachable and readable by your NiFi's service user.  Then as commented above configure the "Database Drivers Location(s)" property in the DBCPConnectionPool controller service to point to that new directory.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#6]

@Jim_Steinebrey     @MattWho 

Thanks for your response.  As mentioned, I have created another folder "thirdparty_lib" and updated the new location of jar file in "Database Driver Location(s)" field.  I am still getting the same error.  Please advise.  

[#7]

@shiva239 

Is it the exact same error?
What is the full stack trace logged to the nifi-app.log?

Thanks,
Matt

[#8]

@MattWho   Here is the log from nifi-app.log

2024-04-17 12:50:13,061 ERROR [Timer-Driven Process Thread-2] o.a.n.c.s.StandardControllerServiceNode StandardControllerServiceNode[service=DBCPConnectionPool[id=025a6b87-e362-3761-5695-a2871b444c4b], name=Ignite-db-localhost, active=true] Failed to invoke @OnEnabled method
java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource
	at org.apache.nifi.dbcp.AbstractDBCPConnectionPool.onConfigured(AbstractDBCPConnectionPool.java:139)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
	at org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:655)
	at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-2"]
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:534)
	at java.base/java.lang.Class.forName(Class.java:513)
	at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:559)
	at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:551)
	at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:452)
	at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:429)
	at org.apache.commons.dbcp2.BasicDataSource.<clinit>(BasicDataSource.java:75)
	... 15 common frames omitted

[#9 · Accepted solution]

@shiva239 

As i dig in a bit more here, this looks like a Java version compatibility issue with this Ignite driver.  Apache NiFi 2.0.0-M2 requires Java 21.  

I see other reporting similar:
"Could not initialize class org.apache.ignite.IgniteJdbcThinDriver"

Exceptions when using Java version 16 and newer here:
https://issues.apache.org/jira/browse/IGNITE-14888

I am not familiar with Ignite and its core driver dependencies, but above is likely your issue.  In the above ignite jira, a commentor seemed to work around issue by:

copied the jvm parameters from jvmdefaults.bat in  ignite home catalog

 
Perhaps you can try doing the same and adding those additional jvm paramaters to the NiFi bootstrap.conf file.   This is not something I have ever tried or have an environment in which to test, but perhaps it will help you.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#10]

Thank you @MattWho   Your recommendation worked for me.

I have updated bootstrap.conf file in nifi.  Was able to successfully enable DB controller service and persist data into Ignite database from Nifi.

Details:

Copied jvm parameters available in the file \apache-ignite-2.16.0-bin\bin\include\jvmdefaults.sh to \nifi-2.0.0-M2\conf\bootstrap.conf file

Here is the format and exact list of java arguments added in nifi bootstrap

java.arg.21=--add-opens=java.base/jdk.internal.access=ALL-UNNAMED
java.arg.22=--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
java.arg.23=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
java.arg.24=--add-opens=java.base/sun.util.calendar=ALL-UNNAMED
java.arg.25=--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
java.arg.26=--add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
java.arg.27=--add-opens=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED
java.arg.28=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
java.arg.29=--add-opens=java.base/java.io=ALL-UNNAMED
java.arg.30=--add-opens=java.base/java.nio=ALL-UNNAMED
java.arg.31=--add-opens=java.base/java.net=ALL-UNNAMED
java.arg.32=--add-opens=java.base/java.util=ALL-UNNAMED
java.arg.33=--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
java.arg.34=--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
java.arg.35=--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
java.arg.36=--add-opens=java.base/java.lang=ALL-UNNAMED
java.arg.37=--add-opens=java.base/java.lang.invoke=ALL-UNNAMED
java.arg.38=--add-opens=java.base/java.math=ALL-UNNAMED
java.arg.39=--add-opens=java.sql/java.sql=ALL-UNNAMED
java.arg.40=--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
java.arg.41=--add-opens=java.base/java.time=ALL-UNNAMED
java.arg.42=--add-opens=java.base/java.text=ALL-UNNAMED
java.arg.43=--add-opens=java.management/sun.management=ALL-UNNAMED
java.arg.44=--add-opens=java.desktop/java.awt.font=ALL-UNNAMED

We just need to make sure java.arg.<>  numbers are unused in the bootstrap file that we are working on.

Thanks again!