fsimage missing quota information

[#1]

All,

I would like to generate a list of all HDFS directories for which a quota has been set, and report the quota size.

I have used dfs -count successfully from the command line - the shortcoming is, it is expensive, and running it recursively on every folder of a large HDFS Production cluster is probably not a good idea.

I tried a different approach using fsimage, and oiv to Delimited format. However, the namespace quota and diskspace quota values are consistently -1, 0 or blank. I cannot seem to get the quota value anywhere.

If you have pointers to why this is happening, or an alternative approach to achieve this, I'd love to hear it 🙂

Kindest regards,

-Regis

[#2 · Accepted solution]

My bad, the example pig script in the apache example assumed that quota is int - in my case, it's a long. changing the type fixed the issue.