Skip to main content
shiva239
shiva239
Gold ⭐️
April 4, 2024
Solved

Apache Nifi 2.0-M2 integration with Apache Ignite 2.16 database - Exception java.lang.NoClassDefFoundError

Hi,

I have Nifi 2.0-M2 and Apache Ignite 2.16 database running on my localhost.

http://localhost:8443/nifi

jdbc:ignite:thin://localhost:10800  (I am able to connect to Ignite localhost db from Dbeaver)

I have copied ignite-core-2.16.0.jar file to nifi lib directory (before starting nifi) and tried to create DBCPConnectionPool controller service in Nifi to connect to Ignite database.  

DB Controller service returns the below error.

Failed to invoke @OnEnabled method: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource - Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-10"]

Can you let me know what am I missing here?  Please let me know if more details are required.

Screenshot of DB controller service

shiva239_0-1712268390754.png

 

 

 

 

 

    MattWho
    MattWhoAnswer
    Titanium ⭐️⭐️⭐️
    April 17, 2024

    @shiva239 

    As i dig in a bit more here, this looks like a Java version compatibility issue with this Ignite driver.  Apache NiFi 2.0.0-M2 requires Java 21.  

    I see other reporting similar:
    "Could not initialize class org.apache.ignite.IgniteJdbcThinDriver"

    Exceptions when using Java version 16 and newer here:
    https://issues.apache.org/jira/browse/IGNITE-14888

    I am not familiar with Ignite and its core driver dependencies, but above is likely your issue.  In the above ignite jira, a commentor seemed to work around issue by:

    copied the jvm parameters from jvmdefaults.bat in  ignite home catalog

     
    Perhaps you can try doing the same and adding those additional jvm paramaters to the NiFi bootstrap.conf file.   This is not something I have ever tried or have an environment in which to test, but perhaps it will help you.

    Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

    Thank you,
    Matt

    View in thread

    9 replies

    ckumar
    Platinum ⭐️
    ckumar
    Platinum ⭐️
    April 15, 2024

    Did you update ignite-core-2.16.0.jar file location path under the Database driver location? 

    Translate
      shiva239
      shiva239Author
      Gold ⭐️
      April 15, 2024

      @ckumar   Yes I've placed ignite-core-2.16.0.jar file in nifi lib directory. 

      Translate
        Jim_Steinebrey
        Jim_Steinebrey
        Gold ⭐️
        April 16, 2024

        The third property of the DBCPConnectionPool called "Database Driver Location" needs to be filled with the path to your nifi lib directory and then it should successfully load the JDBC driver class.

        Translate
          MattWho
          Titanium ⭐️⭐️⭐️
          MattWho
          Titanium ⭐️⭐️⭐️
          April 16, 2024

          @Jim_Steinebrey @shiva239 

          I would avoid adding any additional custom jars or nars to the NiFi lib directory.  This will complicate future upgrade efforts.  Instead you should create a new directory outside NiFi's default directory tree for your custom jars that your specific dataflow(s) depend on. Make sure that the custom driver directory is both reachable and readable by your NiFi's service user.  Then as commented above configure the "Database Drivers Location(s)" property in the DBCPConnectionPool controller service to point to that new directory.

          Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

          Thank you,
          Matt

          Translate
          shiva239
          shiva239Author
          Gold ⭐️
          April 16, 2024

          @Jim_Steinebrey     @MattWho 

          Thanks for your response.  As mentioned, I have created another folder "thirdparty_lib" and updated the new location of jar file in "Database Driver Location(s)" field.  I am still getting the same error.  Please advise.  

          shiva239_0-1713299777940.png

           

           

          shiva239_1-1713299157963.png

          Translate
          MattWho
          Titanium ⭐️⭐️⭐️
          MattWho
          Titanium ⭐️⭐️⭐️
          April 17, 2024

          @shiva239 

          Is it the exact same error?
          What is the full stack trace logged to the nifi-app.log?

          Thanks,
          Matt

          Translate
          shiva239
          shiva239Author
          Gold ⭐️
          April 17, 2024

          @MattWho   Here is the log from nifi-app.log

          2024-04-17 12:50:13,061 ERROR [Timer-Driven Process Thread-2] o.a.n.c.s.StandardControllerServiceNode StandardControllerServiceNode[service=DBCPConnectionPool[id=025a6b87-e362-3761-5695-a2871b444c4b], name=Ignite-db-localhost, active=true] Failed to invoke @OnEnabled method
          java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.dbcp2.BasicDataSource
          	at org.apache.nifi.dbcp.AbstractDBCPConnectionPool.onConfigured(AbstractDBCPConnectionPool.java:139)
          	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
          	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
          	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:145)
          	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:133)
          	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:78)
          	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:55)
          	at org.apache.nifi.controller.service.StandardControllerServiceNode$2.run(StandardControllerServiceNode.java:655)
          	at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
          	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
          	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
          	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
          	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
          	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
          	at java.base/java.lang.Thread.run(Thread.java:1583)
          Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: Could not initialize class org.apache.ignite.IgniteJdbcThinDriver [in thread "Timer-Driven Process Thread-2"]
          	at java.base/java.lang.Class.forName0(Native Method)
          	at java.base/java.lang.Class.forName(Class.java:534)
          	at java.base/java.lang.Class.forName(Class.java:513)
          	at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:559)
          	at java.sql/java.sql.DriverManager.isDriverAllowed(DriverManager.java:551)
          	at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:452)
          	at java.sql/java.sql.DriverManager.getDrivers(DriverManager.java:429)
          	at org.apache.commons.dbcp2.BasicDataSource.<clinit>(BasicDataSource.java:75)
          	... 15 common frames omitted

           

          Translate
          MattWho
          Titanium ⭐️⭐️⭐️
          MattWhoAnswer
          Titanium ⭐️⭐️⭐️
          April 17, 2024

          @shiva239 

          As i dig in a bit more here, this looks like a Java version compatibility issue with this Ignite driver.  Apache NiFi 2.0.0-M2 requires Java 21.  

          I see other reporting similar:
          "Could not initialize class org.apache.ignite.IgniteJdbcThinDriver"

          Exceptions when using Java version 16 and newer here:
          https://issues.apache.org/jira/browse/IGNITE-14888

          I am not familiar with Ignite and its core driver dependencies, but above is likely your issue.  In the above ignite jira, a commentor seemed to work around issue by:

          copied the jvm parameters from jvmdefaults.bat in  ignite home catalog

           
          Perhaps you can try doing the same and adding those additional jvm paramaters to the NiFi bootstrap.conf file.   This is not something I have ever tried or have an environment in which to test, but perhaps it will help you.

          Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

          Thank you,
          Matt

          Translate
            shiva239
            shiva239Author
            Gold ⭐️
            April 17, 2024

            Thank you @MattWho   Your recommendation worked for me.

            I have updated bootstrap.conf file in nifi.  Was able to successfully enable DB controller service and persist data into Ignite database from Nifi.

            Details:

            Copied jvm parameters available in the file \apache-ignite-2.16.0-bin\bin\include\jvmdefaults.sh to \nifi-2.0.0-M2\conf\bootstrap.conf file

            Here is the format and exact list of java arguments added in nifi bootstrap

            java.arg.21=--add-opens=java.base/jdk.internal.access=ALL-UNNAMED
            java.arg.22=--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
            java.arg.23=--add-opens=java.base/sun.nio.ch=ALL-UNNAMED
            java.arg.24=--add-opens=java.base/sun.util.calendar=ALL-UNNAMED
            java.arg.25=--add-opens=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
            java.arg.26=--add-opens=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
            java.arg.27=--add-opens=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED
            java.arg.28=--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED
            java.arg.29=--add-opens=java.base/java.io=ALL-UNNAMED
            java.arg.30=--add-opens=java.base/java.nio=ALL-UNNAMED
            java.arg.31=--add-opens=java.base/java.net=ALL-UNNAMED
            java.arg.32=--add-opens=java.base/java.util=ALL-UNNAMED
            java.arg.33=--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
            java.arg.34=--add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
            java.arg.35=--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
            java.arg.36=--add-opens=java.base/java.lang=ALL-UNNAMED
            java.arg.37=--add-opens=java.base/java.lang.invoke=ALL-UNNAMED
            java.arg.38=--add-opens=java.base/java.math=ALL-UNNAMED
            java.arg.39=--add-opens=java.sql/java.sql=ALL-UNNAMED
            java.arg.40=--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
            java.arg.41=--add-opens=java.base/java.time=ALL-UNNAMED
            java.arg.42=--add-opens=java.base/java.text=ALL-UNNAMED
            java.arg.43=--add-opens=java.management/sun.management=ALL-UNNAMED
            java.arg.44=--add-opens=java.desktop/java.awt.font=ALL-UNNAMED

            We just need to make sure java.arg.<>  numbers are unused in the bootstrap file that we are working on.

            Thanks again!

            Translate
              Terms & ConditionsAccessibility statement