I am trying to use a ListSFTP processor to delete files from an SFTP site older than the most recent  "History" file. In order to do this, I need to continually (process would run daily or weekly) query all the files in the SFTP server to see if a new "History" file was added, and if so to remove all files from the site before this newest "History" file. The problem is that ListSFTP only shows the newest files added so I am unable to do this. I can ClearState on the processor to make it work, but I would like to avoid this being a manual process. So how can I have ListSFTP show all the files every single time I run it?   Note: I've seen some posts about automatically clearing the state using a curl request, but this only works on a stopped processor which defeats the purpose of this being an automated process (https://community.cloudera.com/t5/Support-Questions/Nifi-clear-state/td-p/242347) ... View more

I am working off of a 3-node NiFi cluster and that is kicked off by a GenerateFlowfileProcessor run on the primary node, performs some NiFi processing, and then writes the files to the server that I will then run an ExecuteStreamCommand Python script on. The problem I’m running into is I can’t figure out a way to ensure that the processors picking up the first output are run on the same node as the processors that produced the first output.  What is the best way to handle producing files that can be accessed by all nodes? Is there a way to specify the node for a process will be run on? (using “run on primary” is not working as the primary node cycles over the process) ... View more