Why my Ambari Server could not resolve the new IP for the agents even I am using the FQDN for all the hosts?

[#1]

I setup my ambari cluster before by using FQDN to add hosts(add hosts file for all the cluster servers), and due to the local DNS change, we updated the IPs for all the servers in the cluster, and point the server's resolv.conf to the new DNS server, but after i restart all the servers and the ambari, the ambari-UI still shows the old IP address for the hosts.

I can ping the hostname in each server and get the correct IP address responsed.

any idea on this?

[#2]

@Yibing Liu

Have you restarted the ambari Agents? When the agents restarted they send a registration request. As part of this process the new IP/Hostname should also be updated in the database (every time when we restart the agent).

[#3 · Accepted solution]

@Yibing Liu

+1 to @Joy's answer.

Also,

I know you must have taken care of this, however if there are any state entries in /etc/hosts file, please remove that from all the agents.

[#4]

@Yibing Liu You can follow this: While adding the new host to the existing cluster: => Edit the /etc/hosts file on existing Hosts in the cluster and update the ip address and FQDN of the new host on all the nodes.

=> On new host follow the same step and update all the ip address and FQDN of all the host and as well as for the new node.

=> you can save the setting and restart the ambari server and ambari agent.

[#5]

Thanks for all your response, i have resolved this issue, it is related with my local DNS server settings which makes my hostname different with what i am setting for each agent, so the agent could not find the correct hostname and the ip for others.

after i removed my local DNS server and place back the /etc/hosts files to point all the hosts it comes back.

Thanks All.