Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Please help: JAVA HEAP settings for HDFS components lost effects after YARN is enabled.

avatar
Explorer

Before adding YARN services, my JAVA HEAP tunning for datanode, secondarynamenode, and namenode (through cloudera manager WEB UI) are working perfectly.

 

I can see the processes with -Xmx set to my specified value. But after YARN is turned on and cluster is restarted, HDFS node processes' -Xmx value is turned back to 1GB again

 

I checked the cloudera manager Web GUI again and again, and the parameter are still my specified value, say, for namenode it is set at 6GB, but why the when cloudera bring up namenode process, the process comes up with '-Xmx 1GiB', I got completely confused as I am a newbie. Any one see similar issue?

 

I am running Cloudera 5.2.1. Thanks a lot.

1 ACCEPTED SOLUTION

avatar
Explorer

Figured it out. Thanks.

View solution in original post

4 REPLIES 4

avatar
Explorer

Figured it out. Thanks.

avatar
Community Manager

I am happy to see you solved your issue chenggl. Do you mind sharing how you corrected it in case someone else encounters a similar issue?  🙂


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

The problem is in fact, a confusion instead of a problem.

 

HDFS components, say, DataNode, NameNode, etc, runs Java with two -Xmx options. The first one is Hadoop Default, 1GiB,

and the second one is user's custom settings, say 3GiB.

 

When I list the java processes with ps, it has been so difficult to see the second -Xmx option b/c it is buried deep inside the long

Java commands, while the first -Xmx is very easy to browse as it is on the first line (of ps output).

 

I reported the problem and then realized that there was a second -Xmx, and it was supposed to overwrite the first one according to Java

options specifications.

 

In Short, it is not a problem, just a confusion to Cloudera Hadoop newbies. Thanks.

 

avatar
Community Manager

Some of the most usefull solutions result from newbie confusion. Yours may help others more than you imagine. Please mark it as the solution so that others may find it easier. 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.