Support Questions
Find answers, ask questions, and share your expertise

Hello world program of dynamicallyModifiesClasspath() of Apache Nifi properties descriptor not working

Hello world program of dynamicallyModifiesClasspath() of Apache Nifi properties descriptor not working

New Contributor

I have a processor which depends on some jar as a dependency. However changes are made to the dependency jar quite frequenctly. So I want to be able to specify the jar path as a property of processor and let the nifi load the jar each time I modify the path in the property and restart the processor. This is supposed to be doable using dynamicallyModifiesClasspath as explained in this article. However I am unable to do this. Below is my code of hello world program using dynamicallyModifiesClasspath property:

(Below, Djl stands for "dynamic jar loading", a random prefix I guessed to name this hello world program.)

DjlDependencyClass.java

This is the class on which my nifi processor depends and I want to dynamically change its jar path in my nifi processor.

public class DjlDependencyClass {
    public static String getMessage() {
        return "DJL-DEPENDENCY VERSION-1";
    }
}

MyDjlProcessor.java

This is the nifi processor which depends on DjlDependencyClass.

import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.processor.exception.ProcessException;    
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.ProcessorInitializationContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.util.StandardValidators;

import com.mycompany.djldependency.DjlDependencyClass;

import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class MyDjlProcessor extends AbstractProcessor {

    public static final Relationship MY_RELATIONSHIP = new Relationship.Builder()
            .name("MY_RELATIONSHIP")
            .description("Example relationship")
            .build();

    public static final PropertyDescriptor pathToDjlDependencyJar = new PropertyDescriptor.Builder()
            .name("Djl Dependency JAR")
            .description("Djl Dependency JAR")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .dynamicallyModifiesClasspath(true)
            .expressionLanguageSupported(true)
            .build();
   
    private List descriptors;
    private Set relationships;
 
    @Override
    protected void init(final ProcessorInitializationContext context)
    {
     final List descriptors = new ArrayList();
        descriptors.add(pathToDjlDependencyJar);
        this.descriptors = Collections.unmodifiableList(descriptors);
        final Set relationships = new HashSet();
        relationships.add(MY_RELATIONSHIP);
        this.relationships = Collections.unmodifiableSet(relationships);
    }
 
    @Override
    public Set getRelationships() {
     return this.relationships;
    }
 
    @Override
    public final List getSupportedPropertyDescriptors() {
     return descriptors;
    }
 
    @OnScheduled
    public void onScheduled(final ProcessContext context) { }
 
    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException 
    {
     System.out.println(DjlDependencyClass.getMessage());
    }
}

This is how dependency inside pom is marked to have provided scope, so that the dependency jar will not be embedded inside nar:

<dependency>
    <groupId>com.mycompany</groupId>
    <artifactId>djl-dependency</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <scope>provided </scope>
</dependency>

I am testing this processor as follows:

108962-goey01.png

Configuration of the processor:

108977-5jsup1.png

I am getting following exception:

2019-05-27 17:01:54,536 ERROR [Timer-Driven Process Thread-1] com.mycompany.djl.MyDjlProcessor MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19732119332] MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19732119332] failed to process due to java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass; rolling back session: {}
java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass    
at com.mycompany.djl.MyDjlProcessor.onTrigger(MyDjlProcessor.java:76)    
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)    
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)    
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)    
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)    
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)    
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)    
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)    
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)    
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)    
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)    
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)    
at java.lang.Thread.run(Thread.java:748)

Am I doing something stupid here?

Please download both projects (nifi and dependency jar) here.