https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-check-the-total-amount-of-data-present-on-a-hdp/m-p/138958#M35368 <P>How to check the total amount of data present on a hdp Cluster?</P> Thu, 21 Jul 2016 01:58:46 GMT ansarifah 2016-07-21T01:58:46Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-check-the-total-amount-of-data-present-on-a-hdp/m-p/138958#M35368 <P>How to check the total amount of data present on a hdp Cluster?</P> Thu, 21 Jul 2016 01:58:46 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-check-the-total-amount-of-data-present-on-a-hdp/m-p/138958#M35368 ansarifah 2016-07-21T01:58:46Z https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-check-the-total-amount-of-data-present-on-a-hdp/m-p/138959#M35369 <P> <A rel="user" href="https://community.cloudera.com/users/3331/ansarifah.html" nodeid="3331" target="_blank">@ANSARI FAHEEM AHMED</A></P><P><STRONG>1) </STRONG>If you hover your mouse over the "HDFS Disk Usage" widget (upper left hand corner) in the Ambari Dashboard it will show you the following details:</P><P> <STRONG>DFS Used:</STRONG> Storage used for data</P><P> <STRONG>Non-DFS Used:</STRONG> Storage used for things such as logs, shuffle writes, etc...</P><P> <STRONG>Remaining: </STRONG>Remaining storage</P><P> <span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="5904-screen-shot-2016-07-20-at-42443-pm.png" style="width: 1958px;"><img src="https://community.cloudera.com/t5/image/serverpage/image-id/21876i15BE8A0FAE4D6541/image-size/medium?v=v2&amp;px=400" role="button" title="5904-screen-shot-2016-07-20-at-42443-pm.png" alt="5904-screen-shot-2016-07-20-at-42443-pm.png" /></span></P><P><STRONG>2) </STRONG>From the command line you can also run "sudo -u hdfs hdfs dfsadmin -report", which will generate a full report of hdfs storage usage. </P><P><STRONG>3) </STRONG>Finally, if you would like to check the disk usage for a particular folder (and sub folders), then you can use commands like "hadoop fsck", "hadoop fs -dus" or "hadoop fs -count -q". For an explanation of the differences between these commands as well as how to read the results please take a look at this post:</P><P> <A href="http://www.michael-noll.com/blog/2011/10/20/understanding-hdfs-quotas-and-hadoop-fs-and-fsck-tools/" target="_blank" rel="nofollow noopener noreferrer">http://www.michael-noll.com/blog/2011/10/20/understanding-hdfs-quotas-and-hadoop-fs-and-fsck-tools/</A></P> Mon, 19 Aug 2019 08:33:36 GMT https://community.cloudera.com/t5/Archives-of-Support-Questions/How-to-check-the-total-amount-of-data-present-on-a-hdp/m-p/138959#M35369 egarelnabi 2019-08-19T08:33:36Z