Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi process consuming >100%CPU while executing dataflow

Solved Go to solution

Re: Nifi process consuming >100%CPU while executing dataflow

Master Collaborator

@Gubbi please share you nifi info:

 

How many nodes?

How much ram and how many cores per node?

What is min/max memory setting in NiFi?

 

 

Have you done anything with configuring nifi for performance?  For example

  1. Increasing Min/Max Ram?
  2. Disk Partitioning?
  3. Changing the Max Thread Count in Admin->Controller Services?
  4. Setting Concurrent Tasks in processor scheduling tab?

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

View solution in original post

Highlighted

Re: Nifi process consuming >100%CPU while executing dataflow

Contributor

@stevenmatison  :

  1. Changing the Max Thread Count in Admin->Controller Services? Max Timer Driven--16 , Max Event Driven --1 
  2. Setting Concurrent Tasks in processor scheduling tab? - Set to 1 as default
  3. Increasing Min/Max Ram? - Set to default

Below is my Linux configuration :

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 8
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
Stepping: 4
CPU MHz: 2294.609
BogoMIPS: 4589.21
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 25344K
NUMA node0 CPU(s): 0-7

Highlighted

Re: Nifi process consuming >100%CPU while executing dataflow

Master Collaborator

@Gubbi   I think your ListFile proc is still executing 0 sec.   Reference our private message.

 

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Don't have an account?
Coming from Hortonworks? Activate your account here