@Lanic All of NiFi executes within a single JVM, so it is not possible to kill a single thread within a single JVM. Unfortunately, a restart is required to clear hung threads. Hopefully you have ironed out any misconfiguration in development that lead to hung threads before deploying to production. I understand that naytime a restart is needed can be very annoying. Thanks, Matt A few Hortonworks community forum tips: 1. Use "@<username>" in responses so that user gets pinged; otherwise, user may never know you asked follow-up question. 2. Avoid starting a new "Answer" to respond to an existing answer. Instead click "Add comment". This makes following a thread/line of thought easier. 3. When an "Answer" addresses/solves your question, please select "Accept" beneath that answer. This encourages user participation in this forum. ... View more

@dhieru singh Min number of entries must be set and defaults to 1. That is fine as long as you don't set max num entries. You are correct it is (min number of entries AND min group size) OR max number of entries OR max group size. So either of the "max " settings will force a merge just like max bin age will. ... View more

@dhieru singh The mergeContent processor does not remove FlowFiles from an inbound connection until the actual merge process occurs. When mergeContent runs it allocates queued FlowFiles to one or more bins. While the FlowFiles themselves remain on the queue for tracking purposes, any allocated FlowFiles are now owned by that MergeContent processor. They will not be able to list or delete FlowFiles from this queue while the MergeContent processor is running. In your case you have two NiFi nodes each with 10,000 FlowFiles for a total queue of 20,000 FlowFiles. Keep in mind that each of your nodes is merging only those FlowFiles that exist on its same node. In your current configuration you have two min set: min number entries = 10,000 <-- that has men met on both nodes min group size = 127 MB <-- This has not been met yet (you have about 2.2 MB on each node) Because higher of the two must be met before a bin is eligible to merge. Max number entries = 15,000 <-- you have not reached this on either node. Max group size = 128 MB <-- You have not met this on either node. NiFi will force a merge when either of these is reached. In your case, you would hit 15000 long before you reach 128 MB. There is one more setting you have not set at all. Max bin age = Setting this will force the merge of a bin no matter what any of the above settings are if the age of the bin has exceed this configured value. I always recommend you set this to avoid FlowFiles lingering for exceptionally long periods of time. You could increase the backpressure object threshold on the inbound connection, but the size you would need to in crease it to in order to hit your 127 MB min group size would amount to merging more then 565,000 FlowFiles. Since the MergeContent must hold the attributes for all FlowFiles being merged in heap memory, merging this number of FlowFiles in one operation is likely to result in out of memory errors. In cases like this I recommend doing a two phase merge with multiple MergeContent processors. I would configure first MergeContent to merge based on min (15000) and max (20000) number entries. You should also set a max bin age. You will need to make sure you push up your connection object threshold from 10000 to 20000. The second MergeContent Processor would then be set to merge based on min (120 MB) and max (128 MB) group size. You should also set a max bin age. The result of which will be far less heap usage but the same desired end result. The flow would look something like this: Thanks, Matt ... View more

@Ravi Papisetti My suggestion here would be to make the following changes in your nifi.properties file on all nodes: nifi.cluster.node.protocol.threads=15 nifi.cluster.node.connection.timeout=30 secs nifi.cluster.node.read.timeout= 30 secs This will give nodes longer to respond to change requests before they get dropped by cluster coordinator. You may also want to keep an eye our for any OOM or GC issues on your nodes that may be occurring at the times of making these changes. Thanks, Matt ... View more