Is this a bug of ambari ? this occur when delete a host.

[#1]

[#2]

@Jay SenSharma

[#3 · Accepted solution]

@jack jack

Surely 8/7 is not right.

But i was thinking that When you deleted a host from ambari cluster the host count became 7 , But before deleting the host did you Stop the DataNode running on that host? Else from NameNode perspective the Running DataNodes will be still 8 because that DataNode is still reporting to the NameNode.

.

Because DataNodes are reports to the NameNode, So you should check the NameNode UI to findout the exact number of DataNode.

http://$NAMENODE:50070/dfshealth.html#tab-overview  Find the "Live DataNodes"

.

Ambari simply grabs the LiveDataNode count from the NameNode JMX

http://$NAMENODE:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystemState

.

So please check the Deleted Host if the DataNode process is still running there? If yes then stop it.

# ps -ef | grep DateNode
# echo `cat /var/run/hadoop/hdfs/hadoop-hdfs-datanode.pid`
# ps -ef | grep  `cat /var/run/hadoop/hdfs/hadoop-hdfs-datanode.pid`

######## If the PID is live and running then kill it. ######
# kill -9  `cat /var/run/hadoop/hdfs/hadoop-hdfs-datanode.pid`

.

[#4]

the datanode is stopped before delete the host. but when I see the DataNodes Live through jmx, the datanode is still live. How to solve it ?

[#5]

@jack jack

If the NameNode is returning incorrect data then From Ambari side we can not do much.

Please check the NameNode UI to see if you are finding the problematic DataNode listed there?

http://$NAMENODE_HOST:50070/dfshealth.html#tab-datanode

.

Please check if you see the DataNode name listed in the above URL?

Try restarting the NameNode and then see if it fixes the stale datanode list.

.

[#6]

the NameNode returning is still 8, can we change the dashboard get data from HDFS Summary.Because when I delete host the HDFS summary area display correctly

[#7]

this label is sensitive and correct