Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Maven nar plugin looks to not bundle all the dependancies

avatar
Rising Star

I have a custom NiFi processor that worked great up until me trying use a distributedMapCache. I tried to include it like:

import org.apache.nifi.distributed.cache.client.DistributedMapCacheClient;
...
public class ListBox extends AbstractListProcessor {
...
    public static final PropertyDescriptor DISTRIBUTED_CACHE_SERVICE = new PropertyDescriptor.Builder()
            .name("Distributed Cache Service")
            .description("Specifies the Controller Service that should be used to maintain state about what has been pulled from HDFS so that if a new node "
                    + "begins pulling data, it won't duplicate all of the work that has been done.")
            .required(false)
            .identifiesControllerService(DistributedMapCacheClient.class)
            .build();

But then when I mvn clean install and copy the nar over I get the following error:

java.util.ServiceConfigurationError: org.apache.nifi.processor.Processor: Provider org.hortonworks.processors.boxconnector.ListBox could not be instantiated
        at java.util.ServiceLoader.fail(ServiceLoader.java:232) ~[na:1.8.0_91]
        at java.util.ServiceLoader.access$100(ServiceLoader.java:185) ~[na:1.8.0_91]
        at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) ~[na:1.8.0_91]
        at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) ~[na:1.8.0_91]
        at java.util.ServiceLoader$1.next(ServiceLoader.java:480) ~[na:1.8.0_91]
        at org.apache.nifi.nar.ExtensionManager.loadExtensions(ExtensionManager.java:116) ~[nifi-nar-utils-1.1.0.jar:1.1.0]
        at org.apache.nifi.nar.ExtensionManager.discoverExtensions(ExtensionManager.java:97) ~[nifi-nar-utils-1.1.0.jar:1.1.0]
        at org.apache.nifi.NiFi.<init>(NiFi.java:139) ~[nifi-runtime-1.1.0.jar:1.1.0]
        at org.apache.nifi.NiFi.main(NiFi.java:262) ~[nifi-runtime-1.1.0.jar:1.1.0]
Caused by: java.lang.NoClassDefFoundError: org/apache/nifi/distributed/cache/client/DistributedMapCacheClient

I also have the dependancy configured in my pom.xml file:

<dependency>
    <groupId>org.apache.nifi</groupId>
    <artifactId>nifi-distributed-cache-client-service-api</artifactId>
</dependency>

If I copy over the distributed map cache nar before bundling it works fine. Is there somewhere else I have to list the dependancy to get it bundled into the nar?

1 ACCEPTED SOLUTION

avatar
Master Guru

There is a specific way that processors and controller services are linked together....

Your processor JAR project should have a provided dependency on the API of the controller service, and your processor NAR project should have a NAR dependency on the API NAR.

Check out these resources for examples:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExten...

https://github.com/bbende/nifi-dependency-example

View solution in original post

2 REPLIES 2

avatar
Master Guru

There is a specific way that processors and controller services are linked together....

Your processor JAR project should have a provided dependency on the API of the controller service, and your processor NAR project should have a NAR dependency on the API NAR.

Check out these resources for examples:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExten...

https://github.com/bbende/nifi-dependency-example

avatar
Master Guru

Specifically, in your processor POM (which you list above), Bryan is saying that under that dependency you should have a <scope>provided</scope> line, and in your NAR POM you should include:

<dependency>
  <groupId>org.apache.nifi</groupId>
  <artifactId>nifi-standard-services-api-nar</artifactId>
  <type>nar</type>
</dependency>