question Re: I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api? in Support Questions

I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api?

srijitachaturve — Wed, 01 Nov 2017 19:56:32 GMT

@matt burgess

Re: I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api?

ahadjidj — Sun, 18 Aug 2019 07:59:52 GMT

Hi @sri chaturvedi

You can right click on connection and select empty queue

Re: I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api?

srijitachaturve — Wed, 01 Nov 2017 20:20:14 GMT

Hi, Thanks for the suggestion, but doing this thing for 5o flows is very time consuming, i was looking for an alternative if we can empty the queues of a particular Process group all at once, may be from nifi backend using some api?

Re: I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api?

ahadjidj — Wed, 01 Nov 2017 20:31:33 GMT

@sri chaturvedi

Thanks for the clarification. I misunderstood your question.

You can use NIFI rest API to get the list of your connections with this endpoint /process-groups/{id}/connections

And loop over them and empty each connection with this endpoint /flowfile-queues/{id}/drop-requests

https://nifi.apache.org/docs/nifi-docs/rest-api/index.html

Re: I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api?

ahadjidj — Wed, 01 Nov 2017 20:48:11 GMT

@sri chaturvedi

I just tried the following call on my NiFi:

Get my root PG ID with the following call : http://localhost:8080/nifi-api/process-groups/root/ (it's c2d938e4-015e-1000-7261-ec94e01a500d)
Get all connections of my root PG with their IDs : http://localhost:8080/nifi-api/process-groups/c2d938e4-015e-1000-7261-ec94e01a500d/connections
Drop flow files from on of the connection with ID 774b15d2-015f-1000-d65f-951f21fdcf4d with the call http://localhost:8080/nifi-api/flowfile-queues/774b15d2-015f-1000-d65f-951f21fdcf4d/drop-requests

You can put this logic inside a script and use Curl to call NiFi API.

I hope this helps

Re: I have 50 nifi flows for fetching data from rdbms to hbase, the flows have a lot of data queues up in connections between two processors. is there any method to clear up the queues of multiple flows in one go..may be using some rest api?

srijitachaturve — Thu, 09 Nov 2017 19:17:41 GMT

@Abdelkrim Hadjidj Thanks a lot for your inputs into this, i will try this, i think this will surely solve my issue.