RAM has been upgraded from 64 GB to 256 GB in Hadoop cluster. Now How to utilize the full RAM capacity in HADOOP Cluster for best query performance.

[#1]

@Jay Kumar SenSharma

My HDP cluster has default settings before up gradation and post up gradation i need to change the configuration in HDP to utilize full capacity of the RAM . Where do all i need to change.

Hadoop cluster (5 Data Nodes) details:

Earlier: Each node has 4 TB Hard Disk and 64 GB RAM

Post Upgradation of HD and RAM: Each node has 8 TB Hard Disk and 256 GB RAM.

Thanks in advance ....!!!!!

Kotesh

[#2]

@kotesh banoth

For HDFS disk space alter HDFS configurations

make sure all new data disks are provided under datanode directories.

For computation

You will have to alter yarn configuration -

yarn.scheduler.maximum-allocation-mb

yarn.scheduler.maximum-allocation-vcores

yarn.nodemanager.resource.memory-mb