Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there a command to find the number of blocks pending deletion in HDFS?

Solved Go to solution
Highlighted

Is there a command to find the number of blocks pending deletion in HDFS?

Contributor

I am currently deleting a large amount of small files and am needing to monitor the number of blocks pending deletion so I do not put too much of a load on the cluster. I know this number can be viewed in the NameNode UI, but I would like to run a command instead which I can then use in combination with `watch` so I can get continuous updates. Does such a command exist?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Hi @Josh Nicholson

This works for me, "PendingDeletionBlocks" is what you're looking for in the output if you're looking for pending deletes only.

watch curl --silent http://<your-name-node>:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem

View solution in original post

6 REPLIES 6
Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Hi @Josh Nicholson

This works for me, "PendingDeletionBlocks" is what you're looking for in the output if you're looking for pending deletes only.

watch curl --silent http://<your-name-node>:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem

View solution in original post

Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Contributor

I didn't think of querying the JMX, this works fantastically, thanks!

Actually @Jonathan Sneep I am not able to curl to the JMX as I need authorization. This cluster is kerberized and using SSL. I tried searching around on how to pass credentials but couldn't find anything. Any ideas?

Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Hey @Josh Nicholson
curl --negotiate -u should work, I will test this myself in a bit.

Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Contributor

That worked, thanks.

Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Perfect, glad to hear!

Highlighted

Re: Is there a command to find the number of blocks pending deletion in HDFS?

Explorer

goto Ambari --> HDFS --> QuickLinks --> Master(Active) ... a new page will open ... search "Number of Blocks Pending Deletion"... you will find the pending deletion blocks.. refresh page after 30 sec. counts will be changed... enjoy

Don't have an account?
Coming from Hortonworks? Activate your account here