Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How does NiFi support parallel processing of messages ? Also do we have the threading configuration at processor level ? Does the Nifi flow setting applicable to entire NiFi instance?

Solved Go to solution

How does NiFi support parallel processing of messages ? Also do we have the threading configuration at processor level ? Does the Nifi flow setting applicable to entire NiFi instance?

 
1 ACCEPTED SOLUTION

Accepted Solutions

Re: How does NiFi support parallel processing of messages ? Also do we have the threading configuration at processor level ? Does the Nifi flow setting applicable to entire NiFi instance?

Super Guru

Define parallel processing.

You have many nodes for ingest.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html

Event driven: When this mode is selected, the Processor will be triggered to run by an event, and that event occurs when FlowFiles enter Connections feeding this Processor. This mode is currently considered experimental and is not supported by all Processors. When this mode is selected, the ‘Run schedule’ option is not configurable, as the Processor is not triggered to run periodically but as the result of an event. Additionally, this is the only mode for which the ‘Concurrent tasks’ option can be set to 0. In this case, the number of threads is limited only by the size of the Event-Driven Thread Pool that the administrator has configured.

2 REPLIES 2

Re: How does NiFi support parallel processing of messages ? Also do we have the threading configuration at processor level ? Does the Nifi flow setting applicable to entire NiFi instance?

Super Guru

Define parallel processing.

You have many nodes for ingest.

https://nifi.apache.org/docs/nifi-docs/html/user-guide.html

Event driven: When this mode is selected, the Processor will be triggered to run by an event, and that event occurs when FlowFiles enter Connections feeding this Processor. This mode is currently considered experimental and is not supported by all Processors. When this mode is selected, the ‘Run schedule’ option is not configurable, as the Processor is not triggered to run periodically but as the result of an event. Additionally, this is the only mode for which the ‘Concurrent tasks’ option can be set to 0. In this case, the number of threads is limited only by the size of the Event-Driven Thread Pool that the administrator has configured.

Re: How does NiFi support parallel processing of messages ? Also do we have the threading configuration at processor level ? Does the Nifi flow setting applicable to entire NiFi instance?

Thanks @Timothy Spann for this answer, I was wondering which parameter in the nifi propertiescontrols size of the Event Driven Thread Pool ?

Don't have an account?
Coming from Hortonworks? Activate your account here