Unable to clear Nifi Queue

[#1]

Hi all,

I run on NIFI 1.1.1.

It seems I'm not only to unable to empty queue. It is a bug ?

regards

[#2 · Accepted solution]

@mayki wogno

What type of downstream processor is the queue you are trying to empty connect to?

Some processors such as the MergeContent processor when running have ownership of FlowFiles in the incoming queue.

The MergeContent assigns FlowFiles on its incoming queue(s) to bins. You will not be able to clear the queue of any FlowFiles that are currently assigned to a bin.

If you stop the processor downstream of your queue (Processor must show no running threads), can you then successfully empty the queue?

Thanks,

Matt

[#3]

@mayki wogno

If you are a secured NiFi cluster, make sure all you nodes have been granted the "modify the data" access policy for those connections (or the containing process group if connections are inheriting policies). As a authenticated and authorized user, when you make a request logged in to one node, that request is replicated to the other nodes. So the purge of data is being done on your behalf by the node you are currently logged in to. Authorizing your nodes to be able to modify the data should allow you to empty the queue successfully.

Another option is to temporarily set file expiration on the connection to 1 sec so that NiFi purges the queue itself. Just don't forget to change it back to avoid data loss when you don't want purging to occur.

Of course as Bryan had noted, you can always top nifi and delete everything in FlowFile and content repositories to purge all data form you dataflow, but that may not always be desired.

Thanks,

Matt

[#4]

I'm trying to empty queue of connect "to routeOnAttribute from routeOnAttribute" and another connecter to routeOnAttribute to deleteHDFS"

So I've two connectors where i unable to empty even if all processors are stopped.

[#5]

I'm trying to empty connector from "routeToAttribute to routeToAttribute" and another connector from "routeToAttribute to deleteHDFS"

Even if i've stopped all procesors in process group, i'm unable to empty queue.

I did stop nifi and remove all files in *_repository like explain Bryan in another post.

[#6]

@Matt, thanks

All nodes in my cluster have been granted "modify the data", i've already succesfully empty queue.

Next time, I'm trying expiration time.