Support Questions

mike_bronson7 · ‎11-23-2017

in our Amabri cluster we see that: all data nodes are up but in dashboard we see only 3 from 5 are up

so - how / why dashboard see only 3 from 5 ?

what need to check or sync here?

* just want to say that two hosts ( workers machine ) was added recently to the ambari cluster , any way we restart the ambari-agent and reboot these servers , but still the status on dasboard is 3/5

Michael-Bronson

jsensharma · ‎11-23-2017

@Michael Bronson

DataNode Issue:

Ambari Server fetches some informations from the NameNode. Like the DataNode status. So as we see that the NameNode is saying only 3 DataNodes are Live means other 2 DataNodes are not able to communicate properly with the NameNode. Even though those DataNode might be running (means even if the DataNodes will be running and might be having a valid PID file) they are not communicating fine with the NameNode and hence ambari is just showing the information which it is getting from NameNode.

.

So at this point we can say that there is no issue from ambari side and it is showing the info about the Live DataNodes, what it is getting form the NameNode.

.

So in order to investigate why those DataNodes are not communicating fine with nameNode (Why Name Node is not showing all 5 nodes as Live) we will have to look at the NameNode log as well as the DataNode logs of the problematic DataNodes.

.

Regarding the Agent communication with Ambari Server:

Unable to reconnect to https://master02:8441/agent/v1/heartbeat/worker06.sys674.com

.

Please check if those hosts are resolving the Ambari Server hostname & IP Address properly? Pleas check the "/etc/hosts" file entry of those hosts to verify if the ambari host is resolving fine.

Also please check if there is any port blockage OR Firewall issue in communicating to ambari server port 8441 frm those hosts?

# cat /etc/hosts
# nc -v master02 8441
(OR)
# telnet master02 8441

.

Please confirm that the "master02" is actually your Ambari Server host? If not then please check the "/etc/ambari-agent/conf/ambari-agent.ini" file to verify if the Ambari Hostname is correctly mentioned there?

View solution in original post

mike_bronson7 · ‎11-23-2017

linux kernel - 3.10.0-327.el7.x86_64 is the same on all hosts , and the linux version also

Michael-Bronson

mike_bronson7 · ‎11-23-2017

hi Jay , I am really got loss here , what we can do next?

Michael-Bronson