Created on 04-04-2024 03:02 PM - edited 04-04-2024 03:06 PM
Hi,
I have Nifi 2.0-M2 and Apache Ignite 2.16 database running on my localhost.
jdbc:ignite:thin://localhost:10800 (I am able to connect to Ignite localhost db from Dbeaver)
I have copied ignite-core-2.16.0.jar file to nifi lib directory (before starting nifi) and tried to create DBCPConnectionPool controller service in Nifi to connect to Ignite database.
DB Controller service returns the below error.
Failed to invoke @OnEnabled method: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource - Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-10"]
Can you let me know what am I missing here? Please let me know if more details are required.
Screenshot of DB controller service
Created 04-17-2024 11:30 AM
@shiva239
As i dig in a bit more here, this looks like a Java version compatibility issue with this Ignite driver. Apache NiFi 2.0.0-M2 requires Java 21.
I see other reporting similar:
"Could not initialize class org.apache.ignite.IgniteJdbcThinDriver"
Exceptions when using Java version 16 and newer here:
https://issues.apache.org/jira/browse/IGNITE-14888
I am not familiar with Ignite and its core driver dependencies, but above is likely your issue. In the above ignite jira, a commentor seemed to work around issue by:
copied the jvm parameters from jvmdefaults.bat in ignite home catalog
Perhaps you can try doing the same and adding those additional jvm paramaters to the NiFi bootstrap.conf file. This is not something I have ever tried or have an environment in which to test, but perhaps it will help you.
Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 04-15-2024 06:26 AM
Did you update ignite-core-2.16.0.jar file location path under the Database driver location?
Created 04-15-2024 04:11 PM
@ckumar Yes I've placed ignite-core-2.16.0.jar file in nifi lib directory.
Created 04-15-2024 07:11 PM
The third property of the DBCPConnectionPool called "Database Driver Location" needs to be filled with the path to your nifi lib directory and then it should successfully load the JDBC driver class.
Created 04-16-2024 05:44 AM
@Jim_Steinebrey @shiva239
I would avoid adding any additional custom jars or nars to the NiFi lib directory. This will complicate future upgrade efforts. Instead you should create a new directory outside NiFi's default directory tree for your custom jars that your specific dataflow(s) depend on. Make sure that the custom driver directory is both reachable and readable by your NiFi's service user. Then as commented above configure the "Database Drivers Location(s)" property in the DBCPConnectionPool controller service to point to that new directory.
Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created on 04-16-2024 01:34 PM - edited 04-16-2024 01:36 PM
Thanks for your response. As mentioned, I have created another folder "thirdparty_lib" and updated the new location of jar file in "Database Driver Location(s)" field. I am still getting the same error. Please advise.
Created 04-17-2024 05:42 AM
Is it the exact same error?
What is the full stack trace logged to the nifi-app.log?
Thanks,
Matt
Created 04-17-2024 10:55 AM
@MattWho Here is the log from nifi-app.log
2024-04-17 12:50:13,061 ERROR [Timer-Driven Process Thread-2] o.a.n.c.s.StandardControllerServiceNode StandardControllerServiceNode[service=DBCPConnectionPool[id=025a6b87-e362-3761-5695-a2871b444c4b], name=Ignite-db-localhost, active=true] Failed to invoke @OnEnabled method
java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource
at org.apache.nifi.dbcp.AbstractDBCPConnectionPool.onConfigured(AbstractDBCPConnectionPool.java:139)
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
at org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:655)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-2"]
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:534)
at java.base/java.lang.Class.forName(Class.java:513)
at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:559)
at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:551)
at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:452)
at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:429)
at org.apache.commons.dbcp2.BasicDataSource.<clinit>(BasicDataSource.java:75)
... 15 common frames omitted
Created 04-17-2024 11:30 AM
@shiva239
As i dig in a bit more here, this looks like a Java version compatibility issue with this Ignite driver. Apache NiFi 2.0.0-M2 requires Java 21.
I see other reporting similar:
"Could not initialize class org.apache.ignite.IgniteJdbcThinDriver"
Exceptions when using Java version 16 and newer here:
https://issues.apache.org/jira/browse/IGNITE-14888
I am not familiar with Ignite and its core driver dependencies, but above is likely your issue. In the above ignite jira, a commentor seemed to work around issue by:
copied the jvm parameters from jvmdefaults.bat in ignite home catalog
Perhaps you can try doing the same and adding those additional jvm paramaters to the NiFi bootstrap.conf file. This is not something I have ever tried or have an environment in which to test, but perhaps it will help you.
Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.
Thank you,
Matt
Created 04-17-2024 02:41 PM
Thank you @MattWho Your recommendation worked for me.
I have updated bootstrap.conf file in nifi. Was able to successfully enable DB controller service and persist data into Ignite database from Nifi.
Details:
Copied jvm parameters available in the file \apache-ignite-2.16.0-bin\bin\include\jvmdefaults.sh to \nifi-2.0.0-M2\conf\bootstrap.conf file
Here is the format and exact list of java arguments added in nifi bootstrap
java.arg.21=--add-opens=java.base/jdk.internal.access=ALL-UNNAMED
java.arg.22=--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
java.arg.23=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
java.arg.24=--add-opens=java.base/sun.util.calendar=ALL-UNNAMED
java.arg.25=--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
java.arg.26=--add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
java.arg.27=--add-opens=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED
java.arg.28=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
java.arg.29=--add-opens=java.base/java.io=ALL-UNNAMED
java.arg.30=--add-opens=java.base/java.nio=ALL-UNNAMED
java.arg.31=--add-opens=java.base/java.net=ALL-UNNAMED
java.arg.32=--add-opens=java.base/java.util=ALL-UNNAMED
java.arg.33=--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
java.arg.34=--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
java.arg.35=--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
java.arg.36=--add-opens=java.base/java.lang=ALL-UNNAMED
java.arg.37=--add-opens=java.base/java.lang.invoke=ALL-UNNAMED
java.arg.38=--add-opens=java.base/java.math=ALL-UNNAMED
java.arg.39=--add-opens=java.sql/java.sql=ALL-UNNAMED
java.arg.40=--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
java.arg.41=--add-opens=java.base/java.time=ALL-UNNAMED
java.arg.42=--add-opens=java.base/java.text=ALL-UNNAMED
java.arg.43=--add-opens=java.management/sun.management=ALL-UNNAMED
java.arg.44=--add-opens=java.desktop/java.awt.font=ALL-UNNAMED
We just need to make sure java.arg.<> numbers are unused in the bootstrap file that we are working on.
Thanks again!