when we write large file to nas, the nifi cpu will increase quickly. I catch some logs when nifi is writing data and stuck. The highest threads are all RUNNABLE and final step is EPollArrayWrapper.epollWait. This may the actual issue that cause cpu heavy usage. As I know, this function should be set timeout parameter to 1ms. Can I change it by nifi configuration or others? and anyone has other different knowledge? As I know, when IO resource is not enough, the thread should be blocked or time waiting and won't cost too many cpu resource.
... View more
Description: Nifi node will be stuck and all processors won't deal with flow file when one processor is processing a large flow file, like PutFile, ExecuteStreamCommand, ConvertRecord. Nifi node will be very very slow and heavy cpu usage. Sometime the cpu usage rate will over 700%. Then the heartbeat may be Disconnected. Heavy cpu usage also can cause other processors stuck and never consume any flow file until the large flow file processing is completed. As I know, the PutFile or ExecuteStreamCommand shouldn't cost too many cpu, it is just executing a cp command in linux. We have tried to increase the total thread count. But it is not any effect. This issue happened after we upgrade the nifi version from 1.8 to 1.12. We can't understand why these processors can cost so many cpu than nifi node will be disconnected from cluster. Anyone can help me? Thanks a lot. Nifi Version: 1.12 Machine capacity: cpu 8 core, memory 64G, Xmx 20G, Nifi node amount 5
... View more