Nifi problems with emply queue?

[#1]

Hi, I'm having trouble with Apache NiFi.

Occasionally, NiFi is not able to erase the flowfiles of a queue.

I stop the two processors that are before and after the queue, then with a right click, I display the menu in which I select 'Emply queue' but it doesn't work.

This worries me a lot because the only way I have found to fix this is to restart NiFi, but this is not viable in a production environment.

Someone please know a way to solve this problem; starting the processor ahead of the queue is not a solution, as the queue stays the same way.

attached images

Greetings

[#2 · Accepted solution]

@Lanic

The reason you cannot empty that queue is because the "PutEmail" processor is still running and own that FlowFile. When you stop a processor it goes into a state of "stopping". In this state NiFi will no longer schedule the processor to run, but it does not interrupt any running threads. The small number in the upper right corner indicates the number of active threads tied to this processor component. Here I see "2" on your putEmail processor. Most likely 1 thread is the running configured processor task and the other thread is the stopping thread. Once these threads have completed, you will be able to empty the queue feeding this processor.

If the thread(s) never release, then you have a hung processor. Sometimes this is the result of a poorly configured processor or associated controller service and other times it may be core related. You will need to inspect several Nifi thread dumps to identify the hung thread and see what it is waiting on. You can get a thread dump by running the following command:

<path to Nifi>/bin/nifi.sh dump <name for dump file>

Thanks,

Matt

[#3]

@Lanic

All of NiFi executes within a single JVM, so it is not possible to kill a single thread within a single JVM. Unfortunately, a restart is required to clear hung threads.

Hopefully you have ironed out any misconfiguration in development that lead to hung threads before deploying to production. I understand that naytime a restart is needed can be very annoying.

Thanks,

Matt

A few Hortonworks community forum tips:

1. Use "@<username>" in responses so that user gets pinged; otherwise, user may never know you asked follow-up question.

2. Avoid starting a new "Answer" to respond to an existing answer. Instead click "Add comment". This makes following a thread/line of thought easier.

3. When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum.

[#4]

@Lanic

-

With release of Apache NiFi 1.7 and HDF 3.2 in mid 2018, the ability in terminate threads still executing on a processor that is in a state of "stopping" is now possible. After changing state of processor from start to stop, you will see processor display red square. You should give the running threads an opportunity to complete their execution. If it appears the processor is just not going to stop (hung threads) you can right click on the processor and select "Terminate" from the context menu displayed as follows:

[#5]

I'll try to look at the dump, see if I get anything clear from them.

I wonder in case the processor is hung up, is there a way to restart it, without having to reboot nifi?

Somehow I would like to be able to put NiFi into production and have an alternative to having to restart the entire service.

Thanks for the help Matt.

[#6]

In Nifi >= 1.7 there is possibility to 'Terminate' processor, it helps in such situations:

https://pierrevillard.com/2018/07/02/nifi-1-7-terminate-threads/