@Paul Yang What you have here is very light data flow based on the picture shown. The NiFi RPG will send data in batches of up to 100 for efficiency. So if the input queue has less then 100 files in it when it runs, all of those FlowFile will be routed to a single Node. On next run the next batch would go to a different node. Over time if the dataflow rate is constant, the data should be balanced across your nodes. If i am understanding what you have here, you are feeding the RPG that feeds an input port. That input port feeds an output port. Then you can use various RPGs anywhere in your flow to pull data from that output port. correct? The problem with this is that the RPG runs on every Node. so when a node connects he will try to pull all the files he sees on that connection. Nodes are not aware of how many nodes exist in its cluster and will not say I should only pull x amount so the other nodes can pull the same. Each node acts in a a vacuum and pulls as much data as fast as it can from the output port. I would suggest instead having your remote input port (root level input port) feed its success relationship multiple times in the various sub process groups owned by your various departments. Not only will this provide a better load-balanced delivery of data in the cluster, but it will also improve performance. Thanks, Matt ... View more

If after adding "modify the data" policy it still does not work, check the nifi-user.log to see what entity it is having permissions problems with? Did you set processor level policies on the processors on each side of this queued connection? ... View more

@mayki wogno view the data will give you the ability to list the queue, but will not give you the ability to empty the queue. You need to give yoru nodes and the user making teh request teh ability to "modify teh data" as well. ... View more

@Paul Yang I am a little confused by the screen shot you have provided. You have your GenerateTableFetch processor connected to a Remote Process Group (RPG) which is feeding that an input port more_table_input_port... but you also have that same RPG connected to your ExecuteSQL processor. You flow should look more like this: The RPG will distribute data smartly based on reported load on each node in your cluster. Thanks, Matt ... View more

@sanaz Janbakhsh Please accept the answer that helped you. ... View more

NiFi sets no specific requirement for what user account is used to run NiFi. as long as that user has the ability to read and write to the various NiFi directories, you will be good. what ever user you decide on will need the execute permissions on the files in the bin directory. The purpose of serving the run.as= property is so that NiFi will start as that set user even if the executable is trigger by root. ... View more

@Paul Yang Are you trying to have every node in your cluster execute the same SQL statements? or are you trying to evenly distribute all the generated SQL statements across your cluster so that every node runs different SQL statements? ... View more