Member since
02-05-2025
2
Posts
0
Kudos Received
0
Solutions
07-21-2025
11:48 AM
We want to move from NiFi 1.28.1 -> NiFi 2.4. We have done most of the prerequisites like using the json flow file not xml but are confused with the deprecation of the the ./tls-toolkit.sh. How are NiFi 2.0+ clusters secured? We are using helm charts.
... View more
Labels:
- Labels:
-
Apache NiFi
02-05-2025
12:27 PM
Hello, We are using a Nifi cluster with 5 nodes. Each node is a machine with 48 cores and 280-300 Gi of available memory. The issue we currently have is Nifi having trouble keeping up when a high number of tasks is required, 1,000,000+ tasks. The files are eventually transferred but errors do pop up in the mean time that seem to be irrelevant and more related to the flow trying to keep up. It's a simple flow with a GetFile > UpdateAttribute > PutSFTP > LogMesssage. I have increased concurrent tasks and batch sizes but this has little to no effect.
... View more
Labels:
- Labels:
-
Apache NiFi