Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Is there a command to find the number of blocks pending deletion in HDFS?

avatar
Expert Contributor

I am currently deleting a large amount of small files and am needing to monitor the number of blocks pending deletion so I do not put too much of a load on the cluster. I know this number can be viewed in the NameNode UI, but I would like to run a command instead which I can then use in combination with `watch` so I can get continuous updates. Does such a command exist?

1 ACCEPTED SOLUTION

avatar

Hi @Josh Nicholson

This works for me, "PendingDeletionBlocks" is what you're looking for in the output if you're looking for pending deletes only.

watch curl --silent http://<your-name-node>:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem

View solution in original post

6 REPLIES 6

avatar

Hi @Josh Nicholson

This works for me, "PendingDeletionBlocks" is what you're looking for in the output if you're looking for pending deletes only.

watch curl --silent http://<your-name-node>:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem

avatar
Expert Contributor

I didn't think of querying the JMX, this works fantastically, thanks!

Actually @Jonathan Sneep I am not able to curl to the JMX as I need authorization. This cluster is kerberized and using SSL. I tried searching around on how to pass credentials but couldn't find anything. Any ideas?

avatar

Hey @Josh Nicholson
curl --negotiate -u should work, I will test this myself in a bit.

avatar
Expert Contributor

That worked, thanks.

avatar

Perfect, glad to hear!

avatar
Explorer

goto Ambari --> HDFS --> QuickLinks --> Master(Active) ... a new page will open ... search "Number of Blocks Pending Deletion"... you will find the pending deletion blocks.. refresh page after 30 sec. counts will be changed... enjoy