HDF 2.1.1: Apache nifi is running at PID () but not responding to ping requests

[#1]

Couple of Days Ago I observed that Nifi was not processing any records and it was like hanged. When I saw the bootstrap.log I saw this message "Apache nifi is running at PID () but not responding to ping requests".

The issue got fixed by restarting NiFi service, however what could be the possible reason for this issue , it happened twice in our cluster. What can be done so that issue dont reoccur again.

Request for comments.

[#2]

@Akash S

Are there any errors or warnings in the nifi-app.log file?

Typically, when users describe NiFi as hung, it is usually a result of thread starvation in the flow. If you look at the upper right hand corner of the UI, you will see

Left click, another menu appears

Select Controller Settings, then this window will appear

The default values are 10 and 5, as shown above.

How many processors are in your overall graph? What volume of data is passing through the graph?

[#3]

Hi @Wynner I have about 300 processors and we always have about 20MB streaming data flowing.

What are the optimum values for Maximum Timer Driven Thread Count and Maximum Event Driven Thread Count? How can I decide how much value I have to set for above two parameters.

[#4]

@Akash S

The optimum value for Maximum Timer Driven Thread count will be based on the number of CPU's your system has. Roughly we recommend between 2 and 4 times the number of CPU. So, if your system has 8 CPUs, assuming it isn't bear metal, then your max timer driven threads can be between 16 and 32. If you have a bare metal system, then you will have more room because of hyper-threading, so the values could be between 32 and 64.

The Event Driven thread value can stay at 5, NiFi will not need more than that, no matter how many processors there are on the graph.