question Nifi Building error when creating a brand new custom processor in Support Questions

Nifi Building error when creating a brand new custom processor

coelacanth — Tue, 20 Feb 2024 19:35:01 GMT

I'm trying to create a new Nifi custom processor.
I'm following instructions given on 4 differents tutorials.
I only create the structure of the processor, I don't even try to modifiy the code and I'm always having the same mvn clean install [ERROR].
Can you please help me to resolve this ?
A huge thanks!!!

OS : Windows 10 Home
Nifi version : 2.0.0-M2
Maven : Apache Maven 3.9.6
JDK : Java version: 21.0.2, vendor: Oracle Corporation, runtime: C:\Program Files\Java\jdk-21
Me version : Java and Maven newbi (0.0.1)

Maven is installed and I have added its bin to the path env var.
Jdk in installed and I have added its bin to the path env var.
JAVA_HOME env var is defined as C:\Program Files\Java\jdk-21, where my jdk is installed.

The error:

[ERROR] Failed to execute goal on project nifi-helloworld-nar: Could not resolve dependencies for project org.apache.nifi:nifi-helloworld-nar:nar:1.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.nifi:nifi-standard-services-api-nar:nar:1.0-SNAPSHOT (absent): org.apache.nifi:nifi-standard-services-api-nar:nar:1.0-SNAPSHOT was not found in https://repository.apache.org/snapshots during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of apache.snapshots has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :nifi-helloworld-nar
What I tried:
I've read a couple of tutorials (4) to create such a processor, they all tell to
mvn archetype:generate
cd created folder
mvn clean install
I've tried mvn clean install -e -X swicthes to try to see if I could understand any obvious mistake...
I've only included the log without -e -X not wanting to put too much informations here...
I'm probably not defining a needed environement variable or something...

Commands I used to (try to) create the custom processor:
mvn archetype:generate
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): 2104: nifi type processor : 1 Choose org.apache.nifi:nifi-processor-bundle-archetype version: 67: 67 = 2.0.0-M2 Define value for property 'artifactBaseName': helloworld Define value for property 'groupId': org.apache.nifi Define value for property 'artifactId': nifi-helloworld-bundle Define value for property 'version' 1.0-SNAPSHOT: : Define value for property 'package' org.apache.nifi.processors.helloworld: : Confirm properties configuration: artifactBaseName: helloworld nifiVersion: 2.0.0-M2 groupId: org.apache.nifi artifactId: nifi-helloworld-bundle version: 1.0-SNAPSHOT package: org.apache.nifi.processors.helloworld Y: : Y
cd nifi-helloworld-bundle
mvn clean install

The log of these commands:

C:\Users\ASUS\Desktop\nifi-custom-processor>mvn archetype:generate
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] >>> archetype:3.2.1:generate (default-cli) > generate-sources @ standalone-pom >>>
[INFO]
[INFO] <<< archetype:3.2.1:generate (default-cli) < generate-sources @ standalone-pom <<<
[INFO]
[INFO]
[INFO] --- archetype:3.2.1:generate (default-cli) @ standalone-pom ---
[INFO] Generating project in Interactive mode
[INFO] No archetype defined. Using maven-archetype-quickstart (org.apache.maven.archetypes:maven-archetype-quickstart:1.0)
Choose archetype:
1: remote -> am.ik.archetype:elm-spring-boot-blank-archetype (Blank multi project for Spring Boot + Elm)
2: remote -> am.ik.archetype:graalvm-blank-archetype (Blank project for GraalVM)
3: remote -> am.ik.archetype:graalvm-springmvc-blank-archetype (Blank project for GraalVM + Spring MVC)
.
.
.
3356: remote -> za.co.absa.hyperdrive:component-archetype_2.11 (-)
3357: remote -> za.co.absa.hyperdrive:component-archetype_2.12 (-)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): 2104: nifi
Choose archetype:
1: remote -> org.apache.nifi:nifi-processor-bundle-archetype (-)
2: remote -> org.apache.nifi:nifi-service-bundle-archetype (-)
Choose a number or apply filter (format: [groupId:]artifactId, case sensitive contains): : 1
Choose org.apache.nifi:nifi-processor-bundle-archetype version:
1: 0.0.2-incubating
2: 0.1.0-incubating
3: 0.2.0-incubating
.
.
.
64: 1.24.0
65: 1.25.0
66: 2.0.0-M1
67: 2.0.0-M2
Choose a number: 67:
Define value for property 'artifactBaseName': helloworld
[INFO] Using property: nifiVersion = 2.0.0-M2
Define value for property 'groupId': org.apache.nifi
Define value for property 'artifactId': nifi-helloworld-bundle
Define value for property 'version' 1.0-SNAPSHOT: :
Define value for property 'package' org.apache.nifi.processors.helloworld: :
Confirm properties configuration:
artifactBaseName: helloworld
nifiVersion: 2.0.0-M2
groupId: org.apache.nifi
artifactId: nifi-helloworld-bundle
version: 1.0-SNAPSHOT
package: org.apache.nifi.processors.helloworld
Y: : Y
[INFO] ----------------------------------------------------------------------------
[INFO] Using following parameters for creating project from Archetype: nifi-processor-bundle-archetype:2.0.0-M2
[INFO] ----------------------------------------------------------------------------
[INFO] Parameter: groupId, Value: org.apache.nifi
[INFO] Parameter: artifactId, Value: nifi-helloworld-bundle
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[INFO] Parameter: package, Value: org.apache.nifi.processors.helloworld
[INFO] Parameter: packageInPathFormat, Value: org/apache/nifi/processors/helloworld
[INFO] Parameter: package, Value: org.apache.nifi.processors.helloworld
[INFO] Parameter: artifactBaseName, Value: helloworld
[INFO] Parameter: groupId, Value: org.apache.nifi
[INFO] Parameter: artifactId, Value: nifi-helloworld-bundle
[INFO] Parameter: nifiVersion, Value: 2.0.0-M2
[INFO] Parameter: version, Value: 1.0-SNAPSHOT
[WARNING] Don't override file C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors
[WARNING] Don't override file C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors\src\main\java\org\apache\nifi\processors\helloworld
[WARNING] Don't override file C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors\src\test\java\org\apache\nifi\processors\helloworld
[WARNING] Don't override file C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-nar
[INFO] Project created from Archetype in dir: C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:20 min
[INFO] Finished at: 2024-02-20T11:08:03-05:00
[INFO] ------------------------------------------------------------------------

C:\Users\ASUS\Desktop\nifi-custom-processor>dir

2024-02-20 11:08 AM <DIR> .
2024-02-20 11:08 AM <DIR> ..
2024-02-20 11:08 AM <DIR> nifi-helloworld-bundle

C:\Users\ASUS\Desktop\nifi-custom-processor>cd nifi-helloworld-bundle

C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle>mvn clean install
[INFO] Scanning for projects...
[INFO] Inspecting build with total of 3 modules...
[INFO] Installing Nexus Staging features:
[INFO] ... total of 3 executions of maven-deploy-plugin replaced with nexus-staging-maven-plugin
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] nifi-helloworld-bundle [pom]
[INFO] nifi-helloworld-processors [jar]
[INFO] nifi-helloworld-nar [nar]
[INFO]
[INFO] ---------------< org.apache.nifi:nifi-helloworld-bundle >---------------
[INFO] Building nifi-helloworld-bundle 1.0-SNAPSHOT [1/3]
[INFO] from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- clean:3.3.2:clean (default-clean) @ nifi-helloworld-bundle ---
[INFO]
[INFO] --- enforcer:3.4.1:enforce (enforce-maven-version) @ nifi-helloworld-bundle ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 2: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.4.1:enforce (enforce-java-version) @ nifi-helloworld-bundle ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed
[INFO]
[INFO] --- buildnumber:3.2.0:create (default) @ nifi-helloworld-bundle ---
[INFO]
[INFO] --- remote-resources:3.1.0:process (process-resource-bundles) @ nifi-helloworld-bundle ---
[INFO] Preparing remote bundle org.apache.apache.resources:apache-jar-resource-bundle:1.5
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ nifi-helloworld-bundle ---
[INFO] No site descriptor found: nothing to attach.
[INFO]
[INFO] --- install:3.1.1:install (default-install) @ nifi-helloworld-bundle ---
[INFO] Installing C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\pom.xml to C:\Users\ASUS\.m2\repository\org\apache\nifi\nifi-helloworld-bundle\1.0-SNAPSHOT\nifi-helloworld-bundle-1.0-SNAPSHOT.pom
[INFO]
[INFO] -------------< org.apache.nifi:nifi-helloworld-processors >-------------
[INFO] Building nifi-helloworld-processors 1.0-SNAPSHOT [2/3]
[INFO] from nifi-helloworld-processors\pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.3.2:clean (default-clean) @ nifi-helloworld-processors ---
[INFO]
[INFO] --- enforcer:3.4.1:enforce (enforce-maven-version) @ nifi-helloworld-processors ---
[INFO] Rule 0: org.apache.maven.enforcer.rules.BanDuplicatePomDependencyVersions passed
[INFO] Rule 1: org.apache.maven.enforcer.rules.RequireSameVersions passed
[INFO] Rule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps passed
[INFO] Rule 4: org.apache.maven.enforcer.rules.dependency.BannedDependencies passed
[INFO]
[INFO] --- enforcer:3.4.1:enforce (enforce-java-version) @ nifi-helloworld-processors ---
[INFO]
[INFO] --- buildnumber:3.2.0:create (default) @ nifi-helloworld-processors ---
[INFO]
[INFO] --- remote-resources:3.1.0:process (process-resource-bundles) @ nifi-helloworld-processors ---
[INFO] Preparing remote bundle org.apache.apache.resources:apache-jar-resource-bundle:1.5
[INFO] Copying 3 resources from 1 bundle.
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ nifi-helloworld-processors ---
[INFO] Copying 1 resource from src\main\resources to target\classes
[INFO] Copying 3 resources from target\maven-shared-archive-resources to target\classes
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ nifi-helloworld-processors ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 1 source file with javac [forked debug deprecation release 21] to target\classes
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ nifi-helloworld-processors ---
[INFO] skip non existing resourceDirectory C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors\src\test\resources
[INFO] Copying 3 resources from target\maven-shared-archive-resources to target\test-classes
[INFO]
[INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ nifi-helloworld-processors ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 1 source file with javac [forked debug deprecation release 21] to target\test-classes
[INFO]
[INFO] --- surefire:3.1.2:test (default-test) @ nifi-helloworld-processors ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.nifi.processors.helloworld.MyProcessorTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.139 s -- in org.apache.nifi.processors.helloworld.MyProcessorTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- jar:3.3.0:jar (default-jar) @ nifi-helloworld-processors ---
[INFO] Building jar: C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors\target\nifi-helloworld-processors-1.0-SNAPSHOT.jar
[INFO]
[INFO] --- site:3.12.1:attach-descriptor (attach-descriptor) @ nifi-helloworld-processors ---
[INFO] Skipping because packaging 'jar' is not pom.
[INFO]
[INFO] --- install:3.1.1:install (default-install) @ nifi-helloworld-processors ---
[INFO] Installing C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors\pom.xml to C:\Users\ASUS\.m2\repository\org\apache\nifi\nifi-helloworld-processors\1.0-SNAPSHOT\nifi-helloworld-processors-1.0-SNAPSHOT.pom
[INFO] Installing C:\Users\ASUS\Desktop\nifi-custom-processor\nifi-helloworld-bundle\nifi-helloworld-processors\target\nifi-helloworld-processors-1.0-SNAPSHOT.jar to C:\Users\ASUS\.m2\repository\org\apache\nifi\nifi-helloworld-processors\1.0-SNAPSHOT\nifi-helloworld-processors-1.0-SNAPSHOT.jar
[INFO]
[INFO] ----------------< org.apache.nifi:nifi-helloworld-nar >-----------------
[INFO] Building nifi-helloworld-nar 1.0-SNAPSHOT [3/3]
[INFO] from nifi-helloworld-nar\pom.xml
[INFO] --------------------------------[ nar ]---------------------------------
[WARNING] The POM for org.apache.nifi:nifi-standard-services-api-nar:nar:1.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for nifi-helloworld-bundle 1.0-SNAPSHOT:
[INFO]
[INFO] nifi-helloworld-bundle ............................. SUCCESS [ 1.669 s]
[INFO] nifi-helloworld-processors ......................... SUCCESS [ 3.124 s]
[INFO] nifi-helloworld-nar ................................ FAILURE [ 0.036 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.222 s
[INFO] Finished at: 2024-02-20T11:08:59-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project nifi-helloworld-nar: Could not resolve dependencies for project org.apache.nifi:nifi-helloworld-nar:nar:1.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.nifi:nifi-standard-services-api-nar:nar:1.0-SNAPSHOT (absent): org.apache.nifi:nifi-standard-services-api-nar:nar:1.0-SNAPSHOT was not found in https://repository.apache.org/snapshots during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of apache.snapshots has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :nifi-helloworld-nar

Re: Nifi Building error when creating a brand new custom processor

DianaTorres — Tue, 20 Feb 2024 22:23:07 GMT

@coelacanth Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @joseomjr @steven-matison who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Re: Nifi Building error when creating a brand new custom processor

joseomjr — Tue, 20 Feb 2024 22:58:23 GMT

Ultimately, what is it you'll want your custom processor to do? I can tell you a Groovy based custom processor can perform amazingly well and leverage Java libraries just like a native .nar file with the advantage that you can make your changes and test without having to build every time you make a change.

Re: Nifi Building error when creating a brand new custom processor

coelacanth — Wed, 21 Feb 2024 03:18:43 GMT

Goal
At my work (academic field), we are actually testing Nifi and its features to see if we'll adopt it as a workflow solution.
Being able to write our own custom processors would be a good thing.

Groovy processor
Am I right to think you are talking about the ExecuteGroovyScript processor ?
We have two worries about it.

First that it would not be as efficient as a Java one.
But your comment informs me it 's not the case.

Second, this processor is qualified as "experimental" in the documentation.
Is it a good practice to use it anyway in a production environment ?
Is its definition, functionality "stable" or it could change in the future ?

Conclusion
I would still like to be able to create a custom processor in Java 🙂
But I'll sure try the Groovy one 🙂

Thanks a lot!

Re: Nifi Building error when creating a brand new custom processor

coelacanth — Wed, 21 Feb 2024 02:48:09 GMT

Thanks Diana 🙂

Re: Nifi Building error when creating a brand new custom processor

joseomjr — Wed, 21 Feb 2024 21:21:01 GMT

Groovy code can be used with ExecuteGroovyScript or InvokeScriptedProcessor...the latter offers the same structure and functionality as a native NAR file. I've built a production processor that's used by several customers that can easily handle hundreds of millions of events on a single NiFi instance.

NiFi 2.X remove all scripting languages from InvokeScriptedProcessor except for Groovy.

Last time I built a NAR I followed these instructions until I discovered InvokeScriptedProcessor Groovy code gave me the same access and functionality and haven't looked back since.

Creating Custom Processors and Controllers in Apache NiFi | by Hashmap, an NTT DATA Company | Hashmap, an NTT DATA Company | Medium

Re: Nifi Building error when creating a brand new custom processor

coelacanth — Wed, 21 Feb 2024 22:08:18 GMT

Thanks for the answer.

Just to conclude, I'd like to read your toughts in a few words:
I'm still wondering, can these two processors be used safely in a production environment.
The doc writes Experimental for the two of them. Can their behavior change over time, meaning we would have to rewrite our flows if it happens ?

Re: Nifi Building error when creating a brand new custom processor

joseomjr — Thu, 22 Feb 2024 20:35:21 GMT

Others can chime in, but I personally consider them safe for production environments. I written some processors that work from NiFi version 1.8 all the way to 2.0.0 without issues or need to rewrite them.

Re: Nifi Building error when creating a brand new custom processor

mburgess — Thu, 22 Feb 2024 20:38:38 GMT

I think this is a bug, I'm looking into it. Looks like it's using your 1.0-SNAPSHOT version for NiFi dependencies when it should be using nifiVersion. To get things going, use 2.0.0-M2 for your version for now, then you can go back into the helloworld module and change the version back to what you want.

Re: Nifi Building error when creating a brand new custom processor

coelacanth — Thu, 22 Feb 2024 21:28:21 GMT

Thanks à lot for the answer 🙂

It works fine!

Re: Nifi Building error when creating a brand new custom processor

mburgess — Fri, 23 Feb 2024 19:51:20 GMT

https://issues.apache.org/jira/browse/NIFI-12839

Re: Nifi Building error when creating a brand new custom processor

AlokKumar — Mon, 07 Apr 2025 10:03:55 GMT

hello Sir, I have just setup a basic NiFi custom processor. when i try to build, one of the module gets built but the nar module fails with error

[[1;31mERROR[m] [1;31mRule 3: org.apache.maven.enforcer.rules.dependency.RequireReleaseDeps failed with message:[m

[[1;31mERROR[m] [1;31mDependencies outside of Apache NiFi must not use SNAPSHOT versions[m

[[1;31mERROR[m] [1;31mcom.example:nifi-my_custom_nifi_processor-nar:nar:1.0-SNAPSHOT[m

[[1;31mERROR[m] [1;31m com.example:nifi-my_custom_nifi_processor-processors:jar:1.0-SNAPSHOT <--- is not a release dependency