Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Datanode out of memory

avatar
New Contributor

Hello everyone!

 

My cluster have 4 datanode. Each Datanode has 32G RAM.

Recently, the Datanode has been reported Exception Out Of Memory.

 

I looed up some document, just mentioned namenode memory situations.

 

CDH Manager,Host memory usage chart display physical memory is full, but The role of resident memory less than 1G.

 

i don't know why it took so much of physical memory. 

On the system currently has 12 million blocks

 

 

I have turn off all other services,just run the hdfs service.

1 REPLY 1

avatar
Mentor
You've mentioned the RAM of the machine your DataNode is assigned to run
on, however what is your configured DataNode Java JVM heap size? You could
try raising it by 1 GB from its current value, to resolve this.

Also, what's the entire Out of Memory message, 'cause "unable to create a
new native thread" (or summat) is entirely different than "Java heap space"
in what it implies (nproc limit issue vs. actual heap memory exhaustion).