Member since
02-29-2016
3
Posts
5
Kudos Received
1
Solution
My Accepted Solutions
Title | Views | Posted |
---|---|---|
3486 | 03-04-2016 04:00 PM |
03-04-2016
04:00 PM
2 Kudos
We have found the problem. HA for Namenode works fine. But when we are deploying cluster using blueprint only for Resourcemanager HA then Datanodes and Namenode can't to start because of "too small initial heap size" and "too small heap size" exceptions. After some research we figure out that we had missed several configs in our blueprint: yarn.resourcemanager.webapp.address.rm, yarn.resourcemanager.webapp.https.address.rm, fs.defaultFS, hadoop.registry.zk.quorum. Some of these configs are missing in the example for Resourcemanager HA in https://cwiki.apache.org/confluence/display/AMBARI/Blueprint+Support+for+HA+Clusters
... View more
03-01-2016
10:49 AM
1 Kudo
@Neeraj Sabharwal, thank you for reply. We are creating this cluster and blueprint according to steps in the link you've posted above. The question is why we should manually format namenode? If we create cluster from blueprint then it should start without any extra actions from our side, shouldn't it? Also we cannot understand why defaults configs have incorrect properties if we enable HA (issue 1 in my initial question) Maybe this bug https://hortonworks.jira.com/browse/BUG-40852 is our case?
... View more
02-29-2016
06:21 PM
2 Kudos
We are trying to launch HA for NameNode and ResourceManager by using blueprint. We are using HDP 2.3, Ambari 2.1 and 2.2. There are two issues: 1. Namenodes and DataNodes fail to start because of "too small initial heap size" exception. After some research we figure out that several default configs in hadoop-env.sh file don't have 'm' letter (which means 'megabyte'), namely: XX:NewSize={{namenode_opt_newsize}}, XX:MaxNewSize={{namenode_opt_maxnewsize}}, Xms{{namenode_heapsize}}, Xmx{{namenode_heapsize}}, Xmx{{dtnode_heapsize}}. This problem occurs only when we enable HA. Without HA all these configs have 'm' and all works fine. 2. We explicitly setup wrong default configs that pointed above. But we get this situation: NameNodes and ZKFCs are started successfully but after this they are suddenly stopped. In logs we have "NameNode is not formatted" exception.
... View more
Labels: