Java Heap size warning after fresh install

[#1]

Hello,

I receive the following warning message within the Cloudera Manager, all after a brandy new installation of Cloudera 5:

mgmt: Non-Java Memory
The recommended non-Java memory size is 1.5 GiB, 768.0 MiB less than that is configured.

mgmt: Java Heap Size of Host Monitor in Bytes
The recommended heap size is 256.0 MiB bytes, 206.0 MiB less than that is configured.

mgmt: Non-Java Memory
The recommended non-Java memory size is 1.5 GiB, 768.0 MiB less than that is configured.

mgmt: Java Heap Size of Service Monitor in Bytes
The recommended heap size is 256.0 MiB bytes, 206.0 MiB less than that is configured.

hdfs: Java Heap Size of Namenode in Bytes
Java Heap Size of Namenode in Bytes is recommended to be at least 1GB for every million HDFS blocks. Suggested minimum value: 1073741824

I'm running CentOS 6.5 and my question is how could I change the default Java Heap size on the nodes? Also, I'm sure there's enough RAM on all the nodes, so I need help regarding the Non-Java Memory messages as well. Cheers!

[#2 · Accepted solution]

Hi Zealot,

How much ram does your machine have? How many roles did you assign to that host? How many hosts? If you had 64GB or more of ram, you would not be hitting this problem. Probably 32GB would also be ok, at least in terms of getting rid of these warnings.

It sounds like your machine doesn't really have enough memory to handle the roles assigned to it, so when CM automatically tuned the memory to try and split it between the roles, they ended up being below the minimum recommendations. You can configure your various roles to increase their memory (clicking on each of those messages will lead to the relevant config page), but this will just mean that your roles are trying to use more memory than your host physically can provide, so you'll see performance problems or crashes.

You should probably try to use a larger machine or reduce the number of services in your cluster so that everything has enough memory.

Thanks,

Darren

[#3]

Hey Darren,

Thanks for your quick reply. I'm using a virtual environment for some test purposes and seems like I've set too low RAM to be used. Do you have an idea on what's the minimum recommended memory to be used on the Cloudera Manager host (if we set most of the major/master services in control on 1 host) and respectively on the other nodes members of the cluster (assuming we're installing all of the Cloudera Hadoop distribution services and apps in CDH 5.0)? I think it will be helpful to other users as well.

Cheers and have a great day!