Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

NoClassDefFoundError custom controller using external jars

avatar
Explorer

Hi,

I am running into a problem with deploying a custom controller service to NiFi (I am using the latest version of NIfi 1.12 as of writing this). Now what I am trying to do is to create a custom controller service that would allow me to access a Virtuoso DB instance (similarly to DBCPConnectionPool). This controller relies on 2 external jar files (provided by Virtuoso) that I have placed inside a /libs folder under the module that implements the interface of my service. I have pointed to these jars as dependencies in the pom.xml of that same module and maven is able to import them. Furthermore, I am able to successfully compile and run a JUnit test of my implementation of the service which consists of simply getting a connection and checking if it is open.

The problem arises when I compile my service into a NAR and then deploy that service in NiFi. NiFi is able to detect it but when I try to enable it, it throws a NoClassDefFoundError that is related to these external jars. Now I am considering doing similarly to DBCPConnectionPool and create a property to point to these jars and load them externally, however, I simply do not understand what is causing the problem and I am too curious to let it go, I am also concerned that this approach may not be viable for these jars. So i am curious if someone can point me in the right way.
Here is my folder structure:
image.png

Here is the code I am trying to execute @OnEnabled. which passes JUnit

 

 

 

@OnEnabled
    public void onEnabled(final ConfigurationContext context) {

        getLogger().info("Initializing Virtuoso Repository");

        try {
            ConnectionPoolDataSource dataSource = new VirtuosoConnectionPoolDataSource();
            ((VirtuosoConnectionPoolDataSource)dataSource).setCharset(context.getProperty(REPOSITORY_CHARSET).getValue());
            ((VirtuosoConnectionPoolDataSource)dataSource).setServerName(context.getProperty(REPOSITORY_SERVER).getValue());
            ((VirtuosoConnectionPoolDataSource)dataSource).setInitialPoolSize(Integer.parseInt(context.getProperty(REPOSITORY_CONNECTIONPOOLSIZE).getValue()));
            ((VirtuosoConnectionPoolDataSource)dataSource).setMaxPoolSize(Integer.parseInt(context.getProperty(REPOSITORY_MAXCONNECTIONPOOLSIZE).getValue()));
            repository = new VirtuosoRepository(dataSource, context.getProperty(REPOSITORY_DEFAULTGRAPH).getValue(), true);
            repository.initialize();
        }catch (Exception e){
            getLogger().error("Unable to Initialize Virtuoso Repository", e);

        }

    }

 

 

 

Here is the pom where/how I declare the jar dependencies (This is where I am assuming the error might be)

image.png

And this is the log of the error I am getting when I try to enable it from NiFi:

 

 

 

2020-08-31 13:50:03,567 ERROR [Timer-Driven Process Thread-4] o.a.n.c.s.StandardControllerServiceNode Failed to invoke @OnEnabled method of StandardMyService[id=43ef57e8-0174-1000-843d-d32a07ba02bb] due to java.lang.NoClassDefFoundError: virtuoso/jdbc4/VirtuosoConnectionPoolDataSource
2020-08-31 13:50:03,567 ERROR [Timer-Driven Process Thread-4] e.e.c.dct.Virtuoso.StandardMyService StandardMyService[id=43ef57e8-0174-1000-843d-d32a07ba02bb] Could not Shut Down Repository: java.lang.NullPointerException
java.lang.NullPointerException: null
	at eu.everis.cordis.dct.Virtuoso.StandardMyService.shutdown(StandardMyService.java:187)
	at jdk.internal.reflect.GeneratedMethodAccessor376.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:142)

 

 

 

Hope someone can explain where the problem comes from,
BR,
Tino

1 ACCEPTED SOLUTION

avatar
Explorer

An update since I managed to solve the issue. I followed the advice given in this StackOverflow post https://stackoverflow.com/questions/364114/can-i-add-jars-to-maven-2-build-classpath-without-install... by creating a static in-project repository so that maven can have better access to the jars. It seemed to do the trick.

View solution in original post

1 REPLY 1

avatar
Explorer

An update since I managed to solve the issue. I followed the advice given in this StackOverflow post https://stackoverflow.com/questions/364114/can-i-add-jars-to-maven-2-build-classpath-without-install... by creating a static in-project repository so that maven can have better access to the jars. It seemed to do the trick.