About matthew_gaetano

matthew_gaetano · ‎03-01-2019

The following would purge all queues indiscriminately from NiFi using the python library nipyapi (assuming no auth) import nipyapi nipyapi.config.nifi_config.host = 'http://nifi:8080/nifi-api #Queues can be large - increase timeout nipyapi.config.short_max_wait = 3600 for queue in nipyapi.canvas.list_all_connections(): print("clearing flow files from connection: " + queue.id, end=': ') print("Successful") if nipyapi.canvas.purge_connection(queue.id) else print("Unsuccessful")

matthew_gaetano · ‎11-28-2017

Hi @Abdelkrim Hadjidj For the most part yes, especially since i didn't know how state functioned in XYZ processors. However in this setup the timestamp is always updated regardless of the success or failure of the invokehttp processor. In a failure scenario the poll would potential miss a period of time (and thus any events/data within) so the timestamp can only be updated under a success condition of a request. At the moment the only thing i can think of is using the distributed cache feature, but that seems like overkill ...

matthew_gaetano · ‎11-24-2017

Hello, Have been recently investing a lot of time in learning and using NiFi for a multitude of cases. On in particular we have a hard time understand how to accomplish is pulling events from a remote restful API endpoint with a rolling time window. We have successfully used invokehttp processor to pull the desired events using a specific query that includes a time parameter (in this case its an epoch timestamp). While this is great the timestamp is static and the goal would be to update the timestamp with the time in which the last successful invokhttp request occurred. Does anyone know how this would be accomplished within NiFi? Thanks ~Regards

Online	Offline
Last Visited	‎03-05-2019 06:48 PM

Member Since	‎11-24-2017 03:26 PM
Last Visited	‎03-05-2019 06:48 PM
Posts	6

Cloudera Community

Re: Can we empty all the queues inside a process-g...

Re: NiFi: Rolling Window Restful API Polling

NiFi: Rolling Window Restful API Polling