Member since
02-21-2023
1
Post
0
Kudos Received
0
Solutions
02-21-2023
06:25 AM
Hi, I'm new to Nifi and currently trying to understand how to improve its performance. In this use case I am getting large zip files (30 - 700 GB) from a network share using the FetchFile processor and then unpacking with UnpackContent. Files that are in the 500GB+ range are taking several hours to move onto the unpack stage. Files are eventually make their way through but I'm not sure on the best way to improve performance. The server is beefy with 1TB RAM, plenty of disk space, 40 core CPU. Utilisation across CPU/Mem/Disk/Network I/O all seem under used whilst the processor is running. I've upped the max timer/event thread counts to 80/5 respectively but that doesn't seem to have made any difference. Is tackling the jvm min/max limits a good option to go for next? thanks for any guidance.
... View more
Labels:
- Labels:
-
Apache NiFi