question Re: Is this a bug of ambari ? this occur when delete a host. in Support Questions

Is this a bug of ambari ? this occur when delete a host.

454596169 — Sun, 18 Aug 2019 10:02:54 GMT

Re: Is this a bug of ambari ? this occur when delete a host.

454596169 — Thu, 06 Jul 2017 14:14:09 GMT

@Jay SenSharma

Re: Is this a bug of ambari ? this occur when delete a host.

jsensharma — Thu, 06 Jul 2017 14:21:12 GMT

@jack jack

Surely 8/7 is not right.

But i was thinking that When you deleted a host from ambari cluster the host count became 7 , But before deleting the host did you Stop the DataNode running on that host? Else from NameNode perspective the Running DataNodes will be still 8 because that DataNode is still reporting to the NameNode.

Because DataNodes are reports to the NameNode, So you should check the NameNode UI to findout the exact number of DataNode.

http://$NAMENODE:50070/dfshealth.html#tab-overview  Find the "Live DataNodes"

Ambari simply grabs the LiveDataNode count from the NameNode JMX

http://$NAMENODE:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystemState

So please check the Deleted Host if the DataNode process is still running there? If yes then stop it.

# ps -ef | grep DateNode
# echo `cat /var/run/hadoop/hdfs/hadoop-hdfs-datanode.pid`
# ps -ef | grep  `cat /var/run/hadoop/hdfs/hadoop-hdfs-datanode.pid`

######## If the PID is live and running then kill it. ######
# kill -9  `cat /var/run/hadoop/hdfs/hadoop-hdfs-datanode.pid`

Re: Is this a bug of ambari ? this occur when delete a host.

454596169 — Mon, 17 Jul 2017 16:28:52 GMT

the datanode is stopped before delete the host. but when I see the DataNodes Live through jmx, the datanode is still live. How to solve it ?

Re: Is this a bug of ambari ? this occur when delete a host.

jsensharma — Mon, 17 Jul 2017 16:44:28 GMT

@jack jack

If the NameNode is returning incorrect data then From Ambari side we can not do much.

Please check the NameNode UI to see if you are finding the problematic DataNode listed there?

http://$NAMENODE_HOST:50070/dfshealth.html#tab-datanode

Please check if you see the DataNode name listed in the above URL?

Try restarting the NameNode and then see if it fixes the stale datanode list.

Re: Is this a bug of ambari ? this occur when delete a host.

454596169 — Mon, 17 Jul 2017 16:50:14 GMT

the NameNode returning is still 8, can we change the dashboard get data from HDFS Summary.Because when I delete host the HDFS summary area display correctly

Re: Is this a bug of ambari ? this occur when delete a host.

454596169 — Sun, 18 Aug 2019 10:02:46 GMT

this label is sensitive and correct