Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

"ActivityAnalyzerFacade" consuming high ammount of memory resources for a server and launched by root

Solved Go to solution
Highlighted

"ActivityAnalyzerFacade" consuming high ammount of memory resources for a server and launched by root

Rising Star

We are observing mentioned process is consuming high CPU resources for a particular server and going through the clutser looking for same process, I 've founded on other 2 servers, running under a different user and they are not even close to the consumpltion level like the one I am having the doubt with.

What I want to know is if this is a normal behavior and if I should't worry abou it.

server_1_[root@server_1 ~]# top -p 16193
top - 18:04:58 up 479 days, 21:34,  2 users,  load average: 5.79, 3.53, 2.51
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s): 20.4 us,  4.0 sy,  0.0 ni, 75.1 id,  0.0 wa,  0.0 hi,  0.5 si,  0.0 st
KiB Mem : 26384640+total, 11807649+free, 26398576 used, 11937132+buff/cache
KiB Swap:  4194556 total,  4194556 free,        0 used. 23283172+avail Mem


  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
16193 alipiah   20   0 15.861g 2.012g  30100 S   0.7  0.8  14:42.70 java


**********************************************************************************


server_2_[root@server_2 ~]# top -p 9864
top - 18:06:52 up 479 days, 21:36,  2 users,  load average: 0.39, 0.47, 0.49
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.8 us,  0.3 sy,  0.0 ni, 98.9 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 26384640+total, 66057520 free, 26081160 used, 17170771+buff/cache
KiB Swap:  4194556 total,  4194556 free,        0 used. 23298083+avail Mem


  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 9864 alipiah   20   0 15.857g 438752  30048 S   0.0  0.2   3:12.83 java


**********************************************************************************


server_3_[root@server_3 ~]# top -p 344095
top - 18:08:43 up 479 days, 21:38,  2 users,  load average: 38.80, 35.37, 32.46
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s): 81.3 us,  1.9 sy,  0.0 ni, 16.8 id,  0.0 wa,  0.0 hi,  0.1 si,  0.0 st
KiB Mem : 26384640+total, 19629884 free, 42436532 used, 20177996+buff/cache
KiB Swap:  4194556 total,  4194556 free,        0 used. 21663824+avail Mem


   PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
344095 root      20   0 29.954g 8.888g  23968 S  2129  3.5  82702:17 java
1 ACCEPTED SOLUTION

Accepted Solutions

Re: "ActivityAnalyzerFacade" consuming high ammount of memory resources for a server and launched by root

Cloudera Employee

@Jorge Luis Hernandez Olmos,

We have seen this happen in some versions. There are some Tez analyzers that consume a lot of resources. Do this config change to disable the analyzers that consume more resources. Disabling these does not impact your normal processing or analysis.

Ambari > SmartSenseConfigs> custom activity-conf :

tez_job.activity.analyzers= com.hortonworks.smartsense.activity.job.tez.analyzer.SlowNodeAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.LocalityAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.SkewAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.SlowestVertexAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.SpillAnalyzer

This disables ContainerReuseAnalyzer, SlowTaskIdentifier, CriticalPathAnalyzerandShuffleTimeAnalyzer.

Pls note, by default activity analysis consumes minimum of 8 GB of heap even if above analyzers are disabled. So, it is ok to see such consumption even after disabling these tez analyzers.

View solution in original post

1 REPLY 1

Re: "ActivityAnalyzerFacade" consuming high ammount of memory resources for a server and launched by root

Cloudera Employee

@Jorge Luis Hernandez Olmos,

We have seen this happen in some versions. There are some Tez analyzers that consume a lot of resources. Do this config change to disable the analyzers that consume more resources. Disabling these does not impact your normal processing or analysis.

Ambari > SmartSenseConfigs> custom activity-conf :

tez_job.activity.analyzers= com.hortonworks.smartsense.activity.job.tez.analyzer.SlowNodeAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.LocalityAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.SkewAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.SlowestVertexAnalyzer, com.hortonworks.smartsense.activity.job.tez.analyzer.SpillAnalyzer

This disables ContainerReuseAnalyzer, SlowTaskIdentifier, CriticalPathAnalyzerandShuffleTimeAnalyzer.

Pls note, by default activity analysis consumes minimum of 8 GB of heap even if above analyzers are disabled. So, it is ok to see such consumption even after disabling these tez analyzers.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here