Support Questions
Find answers, ask questions, and share your expertise

Call same NiFi Processor in parallel with different parameters

New Contributor

Hi,

 

Is it possible to call/run the same NiFi Processor e.g. "ExecuteSQL" in parallel wherein the sql query is dynamic i.e. replace the variables in the sql e.g. table name/schema name? How do we differentiate between the 2 processes if they are running in parallel?

 

Thanks,

1 REPLY 1

Master Guru

@HiSunny 

The configured "run schedule" on a NiFi processor controls how often the processor is scheduled to execute. The "concurrent tasks" controls the parallel execution of a processor.  When a processor is scheduled to execute it will request a thread from the NiFi Max Timer Driven thread pool that will be used to execute the processor code.  If that thread is still active upon next scheduled Execution and not all concurrent tasks are in use yet, the processor can request another thread to execute in parallel.  

When it comes to variables, If the processor property support NiFi Expression Language (NEL) with an evaluation using FlowFile attributes, then each FlowFile via FlowFile attributes can provide unique input to the execution.  

You can see if a property supports Expression language by floating your cursor of the "?" icon next to each property name.  You'll want to make sure the scope of NEL support supports FlowFile attributes.


If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post.

Thank you,

Matt