Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎08-30-2016

Containers dropping off during job execution

I have configured a cluster with CDH 5.8.0. I am using CentOS 7.2. Cluster has 11 nodes out of which 2 are master nodes, 4 DN (datanode) +NM (nodemanager), 5 NM nodes. I have 32 core CPU available on each node. I was trying to execute TestDFSIO job on this cluster. When the job starts, 15 containers each from the 4 nodes having DN+NM on them start, 32 containers each from the 5 nodes having NM running on them start. On total, 220 containers start, after executing 32 map tasks each, the 5 nodes do not take up any other map task. All other map tasks are being executed by the 4 nodes with DN+NM running on them. Why is this happening?