Support Questions
Find answers, ask questions, and share your expertise

Hello world program of dynamicallyModifiesClasspath() of Apache Nifi properties descriptor not working

New Contributor

I have a processor which depends on some jar as a dependency. However changes are made to the dependency jar quite frequenctly. So I want to be able to specify the jar path as a property of processor and let the nifi load the jar each time I modify the path in the property and restart the processor. This is supposed to be doable using dynamicallyModifiesClasspath as explained in this article. However I am unable to do this. Below is my code of hello world program using dynamicallyModifiesClasspath property:

(Below, Djl stands for "dynamic jar loading", a random prefix I guessed to name this hello world program.)

DjlDependencyClass.java

This is the class on which my nifi processor depends and I want to dynamically change its jar path in my nifi processor.

public class DjlDependencyClass {
    public static String getMessage() {
        return "DJL-DEPENDENCY VERSION-1";
    }
}

MyDjlProcessor.java

This is the nifi processor which depends on DjlDependencyClass.

import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.processor.exception.ProcessException;    
import org.apache.nifi.processor.AbstractProcessor;
import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.ProcessorInitializationContext;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.util.StandardValidators;

import com.mycompany.djldependency.DjlDependencyClass;

import java.util.ArrayList;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

public class MyDjlProcessor extends AbstractProcessor {

    public static final Relationship MY_RELATIONSHIP = new Relationship.Builder()
            .name("MY_RELATIONSHIP")
            .description("Example relationship")
            .build();

    public static final PropertyDescriptor pathToDjlDependencyJar = new PropertyDescriptor.Builder()
            .name("Djl Dependency JAR")
            .description("Djl Dependency JAR")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .dynamicallyModifiesClasspath(true)
            .expressionLanguageSupported(true)
            .build();
   
    private List descriptors;
    private Set relationships;
 
    @Override
    protected void init(final ProcessorInitializationContext context)
    {
     final List descriptors = new ArrayList();
        descriptors.add(pathToDjlDependencyJar);
        this.descriptors = Collections.unmodifiableList(descriptors);
        final Set relationships = new HashSet();
        relationships.add(MY_RELATIONSHIP);
        this.relationships = Collections.unmodifiableSet(relationships);
    }
 
    @Override
    public Set getRelationships() {
     return this.relationships;
    }
 
    @Override
    public final List getSupportedPropertyDescriptors() {
     return descriptors;
    }
 
    @OnScheduled
    public void onScheduled(final ProcessContext context) { }
 
    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException 
    {
     System.out.println(DjlDependencyClass.getMessage());
    }
}

This is how dependency inside pom is marked to have provided scope, so that the dependency jar will not be embedded inside nar:

<dependency>
    <groupId>com.mycompany</groupId>
    <artifactId>djl-dependency</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <scope>provided </scope>
</dependency>

I am testing this processor as follows:

108962-goey01.png

Configuration of the processor:

108977-5jsup1.png

I am getting following exception:

2019-05-27 17:01:54,536 ERROR [Timer-Driven Process Thread-1] com.mycompany.djl.MyDjlProcessor MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19732119332] MyDjlProcessor[id=f8fa5750-016a-1000-ecc3-c19732119332] failed to process due to java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass; rolling back session: {}
java.lang.NoClassDefFoundError: com/mycompany/djldependency/DjlDependencyClass    
at com.mycompany.djl.MyDjlProcessor.onTrigger(MyDjlProcessor.java:76)    
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)    
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1122)    
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)    
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)    
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:128)    
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)    
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)    
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)    
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)    
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)    
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)    
at java.lang.Thread.run(Thread.java:748)

Am I doing something stupid here?

Please download both projects (nifi and dependency jar) here.