question Re: What does HDFS Storage Capacity Usage (Weekly) alert mean? in Archives of Support Questions (Read Only)

What does HDFS Storage Capacity Usage (Weekly) alert mean?

kabadou_rawia — Thu, 05 Oct 2017 16:34:12 GMT

Hello,
I've got the alert mentioned in "alert.png" for 5 days. I need to understand what does it mean and what are the capacities mentioned in it? 625604663B ? 2992430585B ? 598486117B the limit ?

So, can anyone explain me these fields to know how to resolve this issue?

Note: When, i run $hdfs dfs -df -h with hdfs user i get the following result:

Filesystem :hdfs://ambari-server:8020

Size : 68.7 G

Used: 3.0 G

Available: 29.8 G

Use%: 4%

Re: What does HDFS Storage Capacity Usage (Weekly) alert mean?

jsensharma — Thu, 05 Oct 2017 17:04:26 GMT

@raouia

"HDFS Storage Capacity Usage" : This service-level alert is triggered if the increase in storage capacity usage deviation has grown beyond the specified threshold within a given period. This alert will monitor Daily and Weekly periods.

There are tow types of this script "HDFS Storage Capacity Usage (Daily)" and "HDFS Storage Capacity Usage (Weekely)"

In your screenshot we see it it "HDFS Storage Capacity Usage (Weekely)"

625604664B means almost 625 MB
2992430585B means almost 3000 MB
598486117B which is almost 598 MB  This is average.

By default the "HDFS Storage Capacity Usage (Weekly)" has the following limits set:

Growth Rate CRITICAL: 20%
Growth Rate WARNING: 10%

20% of 3000 MB is = 600 MB (Where as your HDFS growth in a week is 625 which is slightly more than 20% .. about 21%) Hence you see this CRITICAL message.

This alert does not have to do anything with the Total Storage Size Or Available Storage Size.

This alert is mainly concerned about the Weekely Usage Growth Rate of the Storage. You can increase the default "Growth Rate CRITICAL: 20%" if you see that 20% weekely growth in Storage Usage Weekely is normal in your scenario.

Re: What does HDFS Storage Capacity Usage (Weekly) alert mean?

kabadou_rawia — Thu, 05 Oct 2017 18:27:47 GMT

@Jay SenSharma

Thank you for your detailed explanation.