where to add external jars in nifi setup.

[#1]

@mclark @PJ Moutrie @Pierre Villard

I have created a custom processor which accept messages from getKafka processor and store in hadoop data lake. This is data lake is a existing project. having multiple third party jars, config files and scripts. my question is where should deploy this project under nifi setup and how to set classpath for that so that my custom processor can easily access all required jars/classes.

[#2]

How did you develop the custom processor? Have you used NiFi's maven-generated project structure? If yes, the build phase will have generated the NAR file which includes all dependencies and you drop it in NiFi's lib dir next. For external configuration artifacts for your processor, look into having a known location accessible by every node (e.g. http or nas), so the processor can read those. There's work under way around Variable Registry to provide a much better story as well.

[#3]

In addition to @Andrew Grande 's answer, since it is a custom processor you could add a property to specify additional folders, JARs, etc. and have the processor build a classloader using those locations. Then you wouldn't have to worry about a NiFi-relative location, although the location(s) would still need to be accessible by each node running the processors (to Andrew's point).

There are examples of this "modules" property in the ExecuteScript and JoltTransformJSON processors.

[#4]

Hi i am new to nifi and maven. i have the same problem described here... i generated the project using maven and now i want to add some external jars in my project. i have added them in the pom file and after i do the maven install, in the generated nar file i dont see my jars being included in the bundled dependencies folder(exploring the nar file with the file extractor tool). can any one tell me how do i add these jar files? at the moment i get class not found exception as the class is in one my external jar files.

@Matt Burgess

@Andrew Grande

@Ankit Jain

,

hi I am new to Nifi and maven.... I have the same problem described in the question here.... I need to include 3 external jars which I have used in my code.... I generated the maven project and then added my code to it.... included the external jars in eclipse and also added them in the pom file.... but when i do a maven install and i look into the nar file (using the extract tool) i see my jars are not in the bundled dependencies folder. how do i add my jars? pleas help. @Matt Burgess @Andrew Grande @Ankit Jain

[#5]

If you added the correct dependencies in your processor's pom file then they should be getting included in the generated NAR, as Andrew mentioned above. Please verify you have the correct project structure described here:

https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions#MavenProjectsforExten...

There should be a Maven project with two sub-modules, one for the processors JAR where you would put your dependencies in the pom file, and another for building the NAR.