Reply
Expert Contributor
Posts: 82
Registered: ‎02-24-2016

Hive Mapreduce job using 100% resources

I have changed my yarn configurations by following YARN Tuning. Now one of the user submitting Hive job and that job using all the available resources, its not releasing resources to other users, so all other jobs are in queue

 

My Cluster details:

Nodes : 7

CPU Cores: 32

Memory: 64GB

Harddisk: 10 TB

 

Changed parameters

 

yarn.scheduler.minimum-allocation-vcores=1

yarn.scheduler.maximum-allocation-vcores=32

yarn.scheduler.increment-allocation-vcores=1

 

yarn.scheduler.minimum-allocation-mb=1024
yarn.scheduler.maximum-allocation-mb= 64GB
yarn.nodemanager.resource.memory-mb=46GB

 

i did not configured any user limitations