Is there a command to find the number of blocks pending deletion in HDFS?

[#1]

I am currently deleting a large amount of small files and am needing to monitor the number of blocks pending deletion so I do not put too much of a load on the cluster. I know this number can be viewed in the NameNode UI, but I would like to run a command instead which I can then use in combination with `watch` so I can get continuous updates. Does such a command exist?

[#2 · Accepted solution]

Hi @Josh Nicholson

This works for me, "PendingDeletionBlocks" is what you're looking for in the output if you're looking for pending deletes only.

watch curl --silent http://<your-name-node>:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem

[#3]

I didn't think of querying the JMX, this works fantastically, thanks!

Actually @Jonathan Sneep I am not able to curl to the JMX as I need authorization. This cluster is kerberized and using SSL. I tried searching around on how to pass credentials but couldn't find anything. Any ideas?

[#4]

Hey @Josh Nicholson
curl --negotiate -u should work, I will test this myself in a bit.

[#5]

That worked, thanks.

[#6]

Perfect, glad to hear!

[#7]

goto Ambari --> HDFS --> QuickLinks --> Master(Active) ... a new page will open ... search "Number of Blocks Pending Deletion"... you will find the pending deletion blocks.. refresh page after 30 sec. counts will be changed... enjoy