New Contributor
Posts: 1
Registered: ‎02-28-2018

Datanode out of memory

Hello everyone!


My cluster have 4 datanode. Each Datanode has 32G RAM.

Recently, the Datanode has been reported Exception Out Of Memory.


I looed up some document, just mentioned namenode memory situations.


CDH Manager,Host memory usage chart display physical memory is full, but The role of resident memory less than 1G.


i don't know why it took so much of physical memory. 

On the system currently has 12 million blocks



I have turn off all other services,just run the hdfs service.

Posts: 1,608
Kudos: 302
Solutions: 245
Registered: ‎07-31-2013

Re: Datanode out of memory

You've mentioned the RAM of the machine your DataNode is assigned to run
on, however what is your configured DataNode Java JVM heap size? You could
try raising it by 1 GB from its current value, to resolve this.

Also, what's the entire Out of Memory message, 'cause "unable to create a
new native thread" (or summat) is entirely different than "Java heap space"
in what it implies (nproc limit issue vs. actual heap memory exhaustion).