Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Maven nar plugin looks to not bundle all the dependancies

Solved Go to solution
Highlighted

Maven nar plugin looks to not bundle all the dependancies

Contributor

I have a custom NiFi processor that worked great up until me trying use a distributedMapCache. I tried to include it like:

import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient;
...
public class ListBox extends AbstractListProcessor {
...
    public static final PropertyDescriptor DISTRIBUTED_CACHE_SERVICE = new PropertyDescriptor.Builder()
            .name("Distributed Cache Service")
            .description("Specifies the Controller Service that should be used to maintain state about what has been pulled from HDFS so that if a new node "
                    + "begins pulling data, it won't duplicate all of the work that has been done.")
            .required(false)
            .identifiesControllerService(DistributedMapCacheClient.class)
            .build();

But then when I mvn clean install and copy the nar over I get the following error:

java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.hortonworks.processors.boxconnector.ListBox could not be instantiated
        at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_91]
        at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[na:1.8.0_91]
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[na:1.8.0_91]
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_91]
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_91]
        at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:116) ~[nifi-nar-utils-1.1.0.jar:1.1.0]
        at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:97) ~[nifi-nar-utils-1.1.0.jar:1.1.0]
        at org.apache.nifi.NiFi.<init>(NiFi.java:139) ~[nifi-runtime-1.1.0.jar:1.1.0]
        at org.apache.nifi.NiFi.main(NiFi.java:262) ~[nifi-runtime-1.1.0.jar:1.1.0]
Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/distributed/cache/client/DistributedMapCacheClient

I also have the dependancy configured in my pom.xml file:

<dependency>
    <groupId>org.apache.nifi</groupId>
    <artifactId>nifi-distributed-cache-client-service-api</artifactId>
</dependency>

If I copy over the distributed map cache nar before bundling it works fine. Is there somewhere else I have to list the dependancy to get it bundled into the nar?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Maven nar plugin looks to not bundle all the dependancies

There is a specific way that processors and controller services are linked together....

Your processor JAR project should have a provided dependency on the API of the controller service, and your processor NAR project should have a NAR dependency on the API NAR.

Check out these resources for examples:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExten...

https://github.com/bbende/nifi-dependency-example

2 REPLIES 2

Re: Maven nar plugin looks to not bundle all the dependancies

There is a specific way that processors and controller services are linked together....

Your processor JAR project should have a provided dependency on the API of the controller service, and your processor NAR project should have a NAR dependency on the API NAR.

Check out these resources for examples:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExten...

https://github.com/bbende/nifi-dependency-example

Re: Maven nar plugin looks to not bundle all the dependancies

Specifically, in your processor POM (which you list above), Bryan is saying that under that dependency you should have a <scope>provided</scope> line, and in your NAR POM you should include:

<dependency>
  <groupId>org.apache.nifi</groupId>
  <artifactId>nifi-standard-services-api-nar</artifactId>
  <type>nar</type>
</dependency>
Don't have an account?
Coming from Hortonworks? Activate your account here