question Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError in Support Questions

Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

shiva239 — Thu, 04 Apr 2024 22:06:58 GMT

Hi,

I have Nifi 2.0-M2 and Apache Ignite 2.16 database running on my localhost.

jdbc:ignite:thin://localhost:10800 (I am able to connect to Ignite localhost db from Dbeaver)

I have copied ignite-core-2.16.0.jar file to nifi lib directory (before starting nifi) and tried to create DBCPConnectionPool controller service in Nifi to connect to Ignite database.

DB Controller service returns the below error.

Failed to invoke @OnEnabled method: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource - Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-10"]

Can you let me know what am I missing here? Please let me know if more details are required.

Screenshot of DB controller service

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

ckumar — Mon, 15 Apr 2024 13:26:04 GMT

Did you update ignite-core-2.16.0.jar file location path under the Database driver location?

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

shiva239 — Mon, 15 Apr 2024 23:11:14 GMT

@ckumar Yes I've placed ignite-core-2.16.0.jar file in nifi lib directory.

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

Jim_Steinebrey — Tue, 16 Apr 2024 02:11:30 GMT

The third property of the DBCPConnectionPool called "Database Driver Location" needs to be filled with the path to your nifi lib directory and then it should successfully load the JDBC driver class.

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

MattWho — Tue, 16 Apr 2024 12:44:29 GMT

@Jim_Steinebrey @shiva239

I would avoid adding any additional custom jars or nars to the NiFi lib directory. This will complicate future upgrade efforts. Instead you should create a new directory outside NiFi's default directory tree for your custom jars that your specific dataflow(s) depend on. Make sure that the custom driver directory is both reachable and readable by your NiFi's service user. Then as commented above configure the "Database Drivers Location(s)" property in the DBCPConnectionPool controller service to point to that new directory.

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

shiva239 — Tue, 16 Apr 2024 20:36:41 GMT

@Jim_Steinebrey @MattWho

Thanks for your response. As mentioned, I have created another folder "thirdparty_lib" and updated the new location of jar file in "Database Driver Location(s)" field. I am still getting the same error. Please advise.

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

MattWho — Wed, 17 Apr 2024 12:42:43 GMT

@shiva239

Is it the exact same error?
What is the full stack trace logged to the nifi-app.log?

Thanks,
Matt

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

shiva239 — Wed, 17 Apr 2024 17:55:08 GMT

@MattWho Here is the log from nifi-app.log

2024-04-17 12:50:13,061 ERROR [Timer-Driven Process Thread-2] o.a.n.c.s.StandardControllerServiceNode StandardControllerServiceNode[service=DBCPConnectionPool[id=025a6b87-e362-3761-5695-a2871b444c4b], name=Ignite-db-localhost, active=true] Failed to invoke @OnEnabled method java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource at org.apache.nifi.dbcp.AbstractDBCPConnectionPool.onConfigured(AbstractDBCPConnectionPool.java:139) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78) at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55) at org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:655) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-2"] at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:534) at java.base/java.lang.Class.forName(Class.java:513) at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:559) at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:551) at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:452) at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:429) at org.apache.commons.dbcp2.BasicDataSource.<clinit>(BasicDataSource.java:75) ... 15 common frames omitted

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

MattWho — Wed, 17 Apr 2024 18:30:52 GMT

@shiva239

As i dig in a bit more here, this looks like a Java version compatibility issue with this Ignite driver. Apache NiFi 2.0.0-M2 requires Java 21.

I see other reporting similar:
"Could not initialize class org.apache.ignite.IgniteJdbcThinDriver"

Exceptions when using Java version 16 and newer here:
https://issues.apache.org/jira/browse/IGNITE-14888

I am not familiar with Ignite and its core driver dependencies, but above is likely your issue. In the above ignite jira, a commentor seemed to work around issue by:

copied the jvm parameters from jvmdefaults.bat in ignite home catalog

Perhaps you can try doing the same and adding those additional jvm paramaters to the NiFi bootstrap.conf file. This is not something I have ever tried or have an environment in which to test, but perhaps it will help you.

Thank you,
Matt

Re: Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

shiva239 — Wed, 17 Apr 2024 21:41:22 GMT

Thank you @MattWho Your recommendation worked for me.

I have updated bootstrap.conf file in nifi. Was able to successfully enable DB controller service and persist data into Ignite database from Nifi.

Details:

Copied jvm parameters available in the file \apache-ignite-2.16.0-bin\bin\include\jvmdefaults.sh to \nifi-2.0.0-M2\conf\bootstrap.conf file

Here is the format and exact list of java arguments added in nifi bootstrap

java.arg.21=--add-opens=java.base/jdk.internal.access=ALL-UNNAMED java.arg.22=--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED java.arg.23=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED java.arg.24=--add-opens=java.base/sun.util.calendar=ALL-UNNAMED java.arg.25=--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED java.arg.26=--add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED java.arg.27=--add-opens=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED java.arg.28=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED java.arg.29=--add-opens=java.base/java.io=ALL-UNNAMED java.arg.30=--add-opens=java.base/java.nio=ALL-UNNAMED java.arg.31=--add-opens=java.base/java.net=ALL-UNNAMED java.arg.32=--add-opens=java.base/java.util=ALL-UNNAMED java.arg.33=--add-opens=java.base/java.util.concurrent=ALL-UNNAMED java.arg.34=--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED java.arg.35=--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED java.arg.36=--add-opens=java.base/java.lang=ALL-UNNAMED java.arg.37=--add-opens=java.base/java.lang.invoke=ALL-UNNAMED java.arg.38=--add-opens=java.base/java.math=ALL-UNNAMED java.arg.39=--add-opens=java.sql/java.sql=ALL-UNNAMED java.arg.40=--add-opens=java.base/java.lang.reflect=ALL-UNNAMED java.arg.41=--add-opens=java.base/java.time=ALL-UNNAMED java.arg.42=--add-opens=java.base/java.text=ALL-UNNAMED java.arg.43=--add-opens=java.management/sun.management=ALL-UNNAMED java.arg.44=--add-opens=java.desktop/java.awt.font=ALL-UNNAMED

We just need to make sure java.arg.<> numbers are unused in the bootstrap file that we are working on.

Thanks again!