@Gu Gur You just need to make sure the following lines in the new nodes nifi.properties file match the same lines in the nifi.properties files of those nodes already in your cluster. I also recommend setting/updating the following line in the nifi.properties file on every node in your cluster each time you add or remove a node: nifi.cluster.flow.election.max.candidates= It is blank by default which means the UI will be unavailable for 5 minutes whenever the cluster is restarted while NiFi waits for the leader election to complete. Of course all your rest of the "cluster node properties" must be set on your new nodes as well. Any changes to the nifi.properties file will require a NiFi restart to take affect Thanks, Matt ... View more

Is NiFi reporting a lot of Garbage collection going on? Do you have a large number of FlowFiles queued in your dataflow(s)? Does a data provenance query run against your RouteOnAttribute processor show extremely slow processing of FlowFiles or a complete stoppage? Does a restart of NiFi trigger it to process files again? ... View more

@John T If you are using the listSFTP processor before your FetchSFTP processor , it will produce a zero byte flow flowfile for every FlowFile it finds on the target SFTP server. The listSFTP processor has a "File Filter Regex" where you can specify a java regular expression to limit what is returned to just files containing "file123.txt". For example "*file123.txt" The ListSFTP processor also maintains state so that the same files are not listed each time. so only new files containing file123.txt are listed each time it runs. The FetchSFTp processor is designed to return the content of a specific file and insert it as content to the FlowFile that he FetchSFTP processor is running against. Thanks, Matt ... View more

Possibility to run every second or minute. In reality this means run as often as possible using the allowable number of concurrent tasks during the 10th hour of each day. I your case it sounds like it was able to run at least 10 times in that one hour. ... View more

@Raphaël MARY If you are running a NiFi cluster, by default every node in your cluster will be running this getHDFS processor at 10 am each day. This means every node will be getting a copy of the same files and processing them in the same way. If you are running a cluster, considering changing the configuration of your getHDFS processor so it runs on primary node only. ... View more

@Rohit Ravishankar Back pressure thresholds are soft limits only. Backpressure is only applied once that threshold has been met or exceeded and remains in affect until the threshold falls below the configured value. Back pressure only affects the processor feeding the connection. processors downstream from where back pressure is being applied continue to run as scheduled. ... View more

@Dmitro Vasilenko The ConsumeKafka_0_10 processor allows dynamic properties: You can try adding a new dynamic property to your ConsumeKafka_0_10 processor for "max.message.bytes" with a value of 2147483647 to see if that works for you. List of Kafka properties can be found here: http://kafka.apache.org/documentation.html#configuration Thanks, Matt ... View more

@Blake Colson If you found my initial response helpful in answering your question, please accept the answer. ... View more