Hi Matt Sorry its taken a few days to post back, we've been running tests and observing behaviour of the clusters. So we have nifi 2.5.0 on a 3 node cluster on ubuntu, and we run zookeeper as the cluster manager. We have a client Windows 11  machine with nifi 2.5.0 on it that sends data files to the cluster, the cluster processes them and sends acknowledgement files ( one each per data file ) back via Nifi to the windows machine. Its at this point where the acknowledgement files get stuck between the cluster and the client machine. We've done belt and braces baselining of zookeeper - looking at logs, making sure NTP had node times correct, memory correct, looked for errors in logs. That came up clean.  In the nifi cluster its also 3 node, and what we did last night was delete the S2S output port and that seemed to fix it. But only temporarily - the files are getting stuck again this morning.  This port connects to a down stream windows machine with nifi 2.5.0 on it. We have the port opened right up to 100,000 files and 1TB for back pressure. Penalty is 30 seconds. Traffic seems to flow reasonably evenly between all 3 nifi nodes. What we have observed and i don't know if its related, but in our non-clustered lab set up, we observed slow site to site between dockers ( 2 dockers on same ubuntu machine ), and so we had to install nifi onto linux directly which seemed to fix it. Then it slowed down again - so we deleted everything on the canvas and re-created it by importing the canvas backup JSON file we took before wiping everything .It then ran OK and seemed happy enough, with decent throughput at levels we would expect. We also observed files would "disappear" when we had 2 downstream sites connected to the same output port, we think there is a not-so-"round robin" happening, whereby we have files moving to one downstream site and others going to a second site but not in any particular sequence which makes them look like on one site they have "disappeared".  We still have files being stuck again. I'm just keen to get to the bottom of it.  🙂 We   ... View more

Thanks Matt, thats a very useful explanation. ... View more

Hi, I did some further digging and found this post ( below ) which has fixed my problem. https://community.cloudera.com/t5/Support-Questions/No-show-Users-and-Policies-in-Global-Menu/td-p/339127 This prompted some questions on nifi security and how its supposed to function. My setup is as a stand alone nifi instance on windows 10. The main reason I wanted admin privs for my generated user was to be able to create new processors and links etc. So I have a couple of questions please: Once I have created processors etc, should I drop the user back to being a normal user instead of being an admin? Is Nifi normally designed to be (say) LDAP connected ? If I wanted to have multiple users as admins, what is the best way to do this on a stand alone instance, or is this something that perhaps should be via LDAP created groups? I think I may not fully understand the concept behind nifi security and how its supposed to work? I'm new-ish to nifi and am used to windows and its security. Thanks in advance.           ... View more

Hi, Thanks, will do. I did try reading Provenance data via the nifi 2.5.0 API and it seems to fail I installed NIfi on my PC with no modifications and all default settings, and just use the default user and password that's created when you install it. I run this in my local windows CMD window and get   "unknown error" : curl --insecure -X POST -H "Authorization: Bearer <my token> " https://localhost:8443/nifi-api/provenance Then I run this and get :  curl --insecure -X POST -H "Authorization: Bearer <token>" https://localhost:8443/nifi-api/provenance > test.txt In text.txt file I get : "An unexpected error has occurred. Please check the logs for additional details." I just wanted to check I have the syntax right.  I ran other API queries for flows etc and they work fine. What I'm trying to do is get a list of all files that have passed through my small local nifi installation so we can track what files have moved through nifi and when, as a form of monitoring. ... View more