Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Please help: JAVA HEAP settings for HDFS components lost effects after YARN is enabled.

Explorer

Before adding YARN services, my JAVA HEAP tunning for datanode, secondarynamenode, and namenode (through cloudera manager WEB UI) are working perfectly.

 

I can see the processes with -Xmx set to my specified value. But after YARN is turned on and cluster is restarted, HDFS node processes' -Xmx value is turned back to 1GB again

 

I checked the cloudera manager Web GUI again and again, and the parameter are still my specified value, say, for namenode it is set at 6GB, but why the when cloudera bring up namenode process, the process comes up with '-Xmx 1GiB', I got completely confused as I am a newbie. Any one see similar issue?

 

I am running Cloudera 5.2.1. Thanks a lot.

1 ACCEPTED SOLUTION

Explorer

Figured it out. Thanks.

View solution in original post

4 REPLIES 4

Explorer

Figured it out. Thanks.

Community Manager

I am happy to see you solved your issue chenggl. Do you mind sharing how you corrected it in case someone else encounters a similar issue?  🙂


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Explorer

The problem is in fact, a confusion instead of a problem.

 

HDFS components, say, DataNode, NameNode, etc, runs Java with two -Xmx options. The first one is Hadoop Default, 1GiB,

and the second one is user's custom settings, say 3GiB.

 

When I list the java processes with ps, it has been so difficult to see the second -Xmx option b/c it is buried deep inside the long

Java commands, while the first -Xmx is very easy to browse as it is on the first line (of ps output).

 

I reported the problem and then realized that there was a second -Xmx, and it was supposed to overwrite the first one according to Java

options specifications.

 

In Short, it is not a problem, just a confusion to Cloudera Hadoop newbies. Thanks.

 

Community Manager

Some of the most usefull solutions result from newbie confusion. Yours may help others more than you imagine. Please mark it as the solution so that others may find it easier. 


Cy Jervis, Manager, Community Program
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.