Cloudera Community
matthew_gaetano
user rank
New Member

Re: Can we empty all the queues inside a process-g...

by New Member in Support Questions
‎03-01-2019 09:42 PM
‎03-01-2019 09:42 PM
The following would purge all queues indiscriminately from NiFi using the python library nipyapi (assuming no auth) import nipyapi nipyapi.config.nifi_config.host = 'http://nifi:8080/nifi-api #Queues can be large - increase timeout nipyapi.config.short_max_wait = 3600 for queue in nipyapi.canvas.list_all_connections(): print("clearing flow files from connection: " + queue.id, end=': ') print("Successful") if nipyapi.canvas.purge_connection(queue.id) else print("Unsuccessful") ... View more

Re: NiFi: Rolling Window Restful API Polling

by New Member in Support Questions
‎11-28-2017 03:16 PM
‎11-28-2017 03:16 PM
Hi @Abdelkrim Hadjidj For the most part yes, especially since i didn't know how state functioned in XYZ processors. However in this setup the timestamp is always updated regardless of the success or failure of the invokehttp processor. In a failure scenario the poll would potential miss a period of time (and thus any events/data within) so the timestamp can only be updated under a success condition of a request. At the moment the only thing i can think of is using the distributed cache feature, but that seems like overkill ... ... View more

NiFi: Rolling Window Restful API Polling

by New Member in Support Questions
‎11-24-2017 08:04 PM
‎11-24-2017 08:04 PM
Hello, Have been recently investing a lot of time in learning and using NiFi for a multitude of cases. On in particular we have a hard time understand how to accomplish is pulling events from a remote restful API endpoint with a rolling time window. We have successfully used invokehttp processor to pull the desired events using a specific query that includes a time parameter (in this case its an epoch timestamp). While this is great the timestamp is static and the goal would be to update the timestamp with the time in which the last successful invokhttp request occurred. Does anyone know how this would be accomplished within NiFi? Thanks ~Regards ... View more
Labels:
Contact Me
Online
Offline
Last Visited
‎03-05-2019 06:48 PM
Community Statistics
Member Since ‎11-24-2017 03:26 PM
Last Visited ‎03-05-2019 06:48 PM
Posts 6