question Re: Memory Utilization is high unable to find what causing this in Archives of Support Questions (Read Only)

Memory Utilization is high unable to find what causing this

hadoop_ammiredd — Sun, 18 Aug 2019 09:45:13 GMT

Re: Memory Utilization is high unable to find what causing this

smayani — Mon, 30 Oct 2017 23:29:43 GMT

Hi @deepak rathod Click on the Scheduler link on the Resource Manager page. It will tell you what is the resource utilization for each queue and and you can drill down to identify which jobs are consuming total of 279 containers ? Appears that you have 279 containers running with avg. of 12 GB , with total of 3.41 TB memory reserved.

Re: Memory Utilization is high unable to find what causing this

hadoop_ammiredd — Mon, 30 Oct 2017 23:50:12 GMT

Hi @Saumil Mayani i found jobs their container comsumption but unable to drill down to see the memory used by each job. is there any way to drill down to memory consumption level?

Re: Memory Utilization is high unable to find what causing this

smayani — Sun, 18 Aug 2019 09:45:05 GMT

Hi @deepak rathod Could you please share the screen capture of the scheduler page. It should have for each application, Allocated Memory and vCores, allong with Running Containers. you can sort and see which application has the most Allocated Memory MB. Sample attached.

Re: Memory Utilization is high unable to find what causing this

hadoop_ammiredd — Sun, 18 Aug 2019 09:44:57 GMT

This is my scheduler @Saumil Mayani

Re: Memory Utilization is high unable to find what causing this

smayani — Tue, 31 Oct 2017 01:35:56 GMT

Hi @deepak rathod appears that the hdp version you are running, does not have this information on YARN Resource Manager UI. Sample screenshot I attached earlier was from HDP-2.6.2.0-205. You may need to upgrade HDP stack.

Re: Memory Utilization is high unable to find what causing this

hadoop_ammiredd — Tue, 31 Oct 2017 02:03:09 GMT

@Saumil Mayani but my version is Hadoop 2.7.1.2.3.2.0-2950

Re: Memory Utilization is high unable to find what causing this

smayani — Tue, 31 Oct 2017 02:07:16 GMT

Hi @deepak rathod yes, you are using HDP-2.3.2.0. You need to upgrade to HDP-2.6.2.0.

Here is the doc:

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/index.html

https://docs.hortonworks.com/HDPDocuments/Ambari-2.5.2.0/bk_ambari-upgrade/content/ambari_upgrade_guide.html