Please help to understand how to correctly build NAR

[#1]

Hi,

we have issues with standard nifi processors, and have to fix them ourselves. I don't want to delve into details why, please take it as an axiom, imagine that I'm creating something from scratch instead.

So I took nifi 1.4.0 archetype to generate project. I created some custom procesor on my own, which does not have any extra dependencies, and validated, that it works in nifi. All good. Now lets move on to nifi code.

I took ValidateRecord file, and copy-pasted it into our project. I copy pasted also all dependencies from pom.xml module ValidateRecord is in. Our maven project and ValidateRecord maven project has now same parent, same dependencies. Both builds just fine.

When I tried to deploy it, it fails, because of missing dependencies, for example on nifi-record. I build our project using mvn clean install in root project. Built nar file does NOT contain respective jar (nifi-record). When I build nifi, also nifi-standard-processors does not contain this jar. I'd consider it normal, since maven parent prescribes provided. We have this dependency in pom.xml:

<dependency> <groupId>org.apache.nifi</groupId> <artifactId>nifi-record</artifactId> <scope>compile</scope> </dependency>

(we added compile in despair, without it it does not work equally)

Question one: how do you declare dependency in nifi/nar if you want it to be available and nar can be deployed??

Now into more wild stuff:

if I run

mvn nifi-nar:nar
it will generate nar files in all 3 target dirs. In root dir with pom packaging it's pure nonsense, in processor's module, with jar packaging there is nar file, which contains blend of normal build and nar build, and it CONTAINS nifi-record library. And finally in packaging module there is nar without nifi-record; same as from mvn clean install.

To conclude: provided, that I work with official archetype and official source code, while using identical dependencies definition, it's surprising that I'm not able to get working code.

What am I doing wrong? (aside of copypasting, but I don't have any other choice)

Thanks!

[#2]

after immense trial&error rounds, we moved forward and currently face this. If we do not provide this dependency in pom.xml

<dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-record-serialization-service-api</artifactId>
            <scope>compile</scope>
        </dependency>

then nar cannot be deployed, because those interfaces are missing. If we leave out compile, nar cannot be deployed, since interfaces are not on classpath. If we keep compile there, interface will be there, but it will be considered to be different version and implementators should extend our interface defined in out built nar.

Is this a bug or what? I assume I should be able to add just:

<dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-record-serialization-service-api</artifactId>
        </dependency>

or what am I doing wrong?

EDIT: current pom.xms:

• packaging pom does not have anything interresting

• pom.xml of nar module

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
...
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>


    <parent>
        <groupId>???</groupId>
        <artifactId>???</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>


    <artifactId>???</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>nar</packaging>
    <properties>
        <maven.javadoc.skip>true</maven.javadoc.skip>
        <source.skip>true</source.skip>
    </properties>


    <dependencies>


        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-record</artifactId>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-record-serialization-service-api</artifactId>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-schema-registry-service-api</artifactId>
            <scope>compile</scope>
        </dependency>


        <dependency>
            <groupId>cz.embedit</groupId>
            <artifactId>nifi-fdp-processors</artifactId>
            <version>1.0-SNAPSHOT</version>
        </dependency>


    </dependencies>


</project>

• pom.xml of processors module

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  ...
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>


    <parent>
        <groupId>???</groupId>
        <artifactId>???</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>


    <artifactId>???</artifactId>
    <packaging>jar</packaging>


    <dependencies>


      <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-standard-processors</artifactId>
        <version>${nifi.version}</version>
        <scope>provided</scope>
      </dependency>


      <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-record-serialization-service-api</artifactId>
          <scope>compile</scope>
      </dependency>
      <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-avro-record-utils</artifactId>
          <scope>compile</scope>
      </dependency>


      <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-schema-registry-service-api</artifactId>
          <scope>compile</scope>
      </dependency>


      <dependency>
        <groupId>org.apache.nifi</groupId>
        <artifactId>nifi-record</artifactId>
          <scope>compile</scope>
      </dependency>


        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.nifi</groupId>
            <artifactId>nifi-utils</artifactId>
        </dependency>
        <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
        </dependency>


    </dependencies>


</project>

[#3 · Accepted solution]

Your processor's pom.xml should have a provided dependencies on

nifi-record-serialization-service-api

Your NAR's pom.xml should have:

<dependency>
    <groupId>org.apache.nifi</groupId>
    <artifactId>nifi-standard-services-api-nar</artifactId>
    <type>nar</type>
</dependency>

The dependency of type "NAR" in your NAR's pom will create a classpath hierarchy between your NAR and standard services API NAR at runtime. So when the app is running your NAR will have all of the classes from standard services API NAR on the class path.

This is also described here:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExten...

[#4]

Thanks a lot! This helped me, based on this hint everything works as it should. Thanks again.

Martin.