Member since
07-08-2015
11
Posts
0
Kudos Received
3
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
2857 | 08-20-2015 10:08 PM | |
2862 | 07-16-2015 04:47 PM | |
1481 | 07-16-2015 04:41 PM |
08-24-2015
12:46 AM
The problem is in fact, a confusion instead of a problem. HDFS components, say, DataNode, NameNode, etc, runs Java with two -Xmx options. The first one is Hadoop Default, 1GiB, and the second one is user's custom settings, say 3GiB. When I list the java processes with ps, it has been so difficult to see the second -Xmx option b/c it is buried deep inside the long Java commands, while the first -Xmx is very easy to browse as it is on the first line (of ps output). I reported the problem and then realized that there was a second -Xmx, and it was supposed to overwrite the first one according to Java options specifications. In Short, it is not a problem, just a confusion to Cloudera Hadoop newbies. Thanks.
... View more
08-20-2015
10:08 PM
Figured it out. Thanks.
... View more
08-20-2015
07:45 PM
Before adding YARN services, my JAVA HEAP tunning for datanode, secondarynamenode, and namenode (through cloudera manager WEB UI) are working perfectly. I can see the processes with -Xmx set to my specified value. But after YARN is turned on and cluster is restarted, HDFS node processes' -Xmx value is turned back to 1GB again I checked the cloudera manager Web GUI again and again, and the parameter are still my specified value, say, for namenode it is set at 6GB, but why the when cloudera bring up namenode process, the process comes up with '-Xmx 1GiB', I got completely confused as I am a newbie. Any one see similar issue? I am running Cloudera 5.2.1. Thanks a lot.
... View more
Labels:
- Labels:
-
Apache YARN
-
Cloudera Manager
-
HDFS
07-16-2015
04:47 PM
Thanks. That's pretty helpful. I am a newbie to cloudera manager, normally I setup cluster(s) with Web UI step by step. With a new business requirement to create clusters on the fly with REST API, I get completely lost here. What's the Steps -- and APIs to call -- for creation of a cluster, with services, and roles, tunning some settings, and then finally bring services up? 1, create cluster 2, create HDFS, and YARN services 3, assign hosts -- and their roles -- to services 4, tunning a few services parameters 5, bring up services initially 6, check cluster, services, and hosts' health status Do anyone has a working steps, API calling sequences for creating HDFS and YARN services on the fly? Thanks.
... View more
07-16-2015
04:41 PM
Thanks. That's exactly what I need. Best, Guolin
... View more
07-08-2015
07:11 PM
Is there a way to tune settings for cloudera manager managed clusters through command line tools? Normally we did it through web UI (port 7180), and then configure options one by one. This is not working for autmatic deployment as manual interactive configuration is involved. Thanks.
... View more
Labels:
- Labels:
-
Cloudera Manager
07-08-2015
07:04 PM
I have a cloudera cluster installed with the cloudera manager, server and agent RPMs, server and agents are all started up. Normally I'll use firefox/Chrome to connect to port 7180 of the cloudera server host, and then follow the configuration templates to select cluster node, services, roles assignment, and finish configuration job. But the question is: Is there a way for me to specify all the parameters in an input file, and then configure the cluster with command line tools, with the input file(s) as input? Basically I'd like to configure a cluster automatically instead of interaction with Web UI. Any one can shed a light on this? Thanks. Thanks a lot.
... View more
Labels:
- Labels:
-
Cloudera Manager