Adding nodes to an HDP cluster

[#1]

These checklists help ensure that you avoid unexpected problems when adding new hosts to an HDP cluster.

Preparation checklist (what to do before adding the new hosts via the Ambari Add Host wizard):

• Have you followed the guidance in the Ambari installation documentation (e.g. example install doc is here)? It is recommended to follow the instructions for the HDP version that is installed on the cluster (go to http://docs.hortonworks.com).
• Does your file system partitioning scheme, storage options, memory, network, and power configurations match what is used for other nodes on the cluster? It is good to either follow the pattern of what is already on the cluster or to follow recommendations from docs.hortonworks.com (see example doc here). Compare and verify that settings and configurations match, including: THP, selinux, ssh keys, user integration, LDAP/AD setup, SSSD integration, drive layouts, etc.
• Have you verified that JAVA_HOME is set properly with same Java version across the cluster? Having mismatched JDK versions across your nodes may cause failures and should be remediated for optimal cluster operation.
• Do the new nodes have the same version of linux, yum, rpm, scp, python and curl as the pre-existing cluster nodes?
• Do all the /etc/hosts files match and reflect the newly added and existing nodes? Check DNS across the nodes to confirm that host names resolve properly
• Have you enabled NTP on all the newly added nodes and make sure they are in sync with other nodes?
• Have you confirmed that iptables are updated or turned off? If you have iptables or firewall settings for port make sure the ports open for new hosts.
• Have you confirmed that the user that Ambari runs as can communicate with new nodes via passwordless SSH from ambari-server to the new nodes?
• Have you verified that the Ambari agents on the new nodes match the ambari-agent version on the existing nodes?
• If using SSL (for HDFS, YARN, etc.), have you installed certificates on the new nodes?
• In case if you are adding the new nodes pulling them from other cluster, make sure you clear the all the previous installs or reformat is good option.

Post node addition checklist:

• When adding the new hosts to the cluster (see “Adding Hosts to a Cluster” in the Ambari User Guide), have you set the rack properly for the new nodes?
• Do you have a strategy in place for rebalancing the cluster that accounts for cluster load? Consider rebalancing in quiet times or running with low bandwidth and throttled if the cluster load will be high during the rebalancing.
• Have you run smoke tests that start an Application Master on one of the new nodes?