HDP Hostnames on EC2

[#1]

I recently installed HDP 2.3 on a 3 node cluster on Amazon EC2. Things went through relatively smoothly but then for some reason the person who set up the nodes restarted them all, causing them all to come up with different internal hostnames. When Ambari loaded back up, everything was in a funky state since they were looking for hostnames that no longer existed.

What is the best practice for setting up HDP to avoid this? Is there anything I can easily do at this point to recover the cluster I had already set up?

Thanks!

[#2 · Accepted solution]

Ambari uses hostnames as keys for all managed hosts.

You can follow the steps to customize the name of your host in Ambari. In the future, I'd recommend using custom hostnames under your own public domain, but following these steps on the new hosts should get you out of the bind you are in.

Ambari presently does not support changing hostnames dynamically. There is a Jira open for this,https://issues.apache.org/jira/browse/AMBARI-4670. But current trunk has no support for it.

[#3]

@Paul Tamburello: Was this answer helpful? If so, could you please accept it through the HCC? If not, is there some follow-up I could help you with?

[#4]

There has been some work in the community on host rename. Looks like AMBARI-4670 is out dated.

Check this JIRA out as it explains the steps, and there are some limitations/restrictions.

https://issues.apache.org/jira/browse/AMBARI-13507

[#5]

@Paul Tamburello

I am wondering how difficult it will be to change the hostname of these servers back to what they were before.

See this http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-hostname.html

[#6]

Paul Tamburello

You can manage these instances by provisioning tool (eg Chef/Puppet) .

Setup the Nodename as you wish for example , master , slave, edge and use one of the recipes like the below one to set the nodename to hostname. This way whenever you reboot the AWS instances receipes will automatically kicks in if there is some discrepency.

https://supermarket.chef.io/cookbooks/hostnames/versions/0.3.1

Also make sure both reverse and forward lookup works.

Hope this helps.