Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

What does HDFS Storage Capacity Usage (Weekly) alert mean?

avatar

Hello,
I've got the alert mentioned in "alert.png" for 5 days. I need to understand what does it mean and what are the capacities mentioned in it? 625604663B ? 2992430585B ? 598486117B the limit ?

So, can anyone explain me these fields to know how to resolve this issue?

Note: When, i run $hdfs dfs -df -h with hdfs user i get the following result:

Filesystem :hdfs://ambari-server:8020

Size : 68.7 G

Used: 3.0 G

Available: 29.8 G

Use%: 4%


alert.png
1 ACCEPTED SOLUTION

avatar
Master Mentor

@raouia

"HDFS Storage Capacity Usage" : This service-level alert is triggered if the increase in storage capacity usage deviation has grown beyond the specified threshold within a given period. This alert will monitor Daily and Weekly periods.


There are tow types of this script "HDFS Storage Capacity Usage (Daily)" and "HDFS Storage Capacity Usage (Weekely)"


In your screenshot we see it it "HDFS Storage Capacity Usage (Weekely)"

625604664B means almost 625 MB
2992430585B means almost 3000 MB
598486117B which is almost 598 MB  This is average.


By default the "HDFS Storage Capacity Usage (Weekly)" has the following limits set:

Growth Rate CRITICAL: 20%
Growth Rate WARNING: 10% 


20% of 3000 MB is = 600 MB (Where as your HDFS growth in a week is 625 which is slightly more than 20% .. about 21%) Hence you see this CRITICAL message.

.

This alert does not have to do anything with the Total Storage Size Or Available Storage Size.

This alert is mainly concerned about the Weekely Usage Growth Rate of the Storage. You can increase the default "Growth Rate CRITICAL: 20%" if you see that 20% weekely growth in Storage Usage Weekely is normal in your scenario.

View solution in original post

2 REPLIES 2

avatar
Master Mentor

@raouia

"HDFS Storage Capacity Usage" : This service-level alert is triggered if the increase in storage capacity usage deviation has grown beyond the specified threshold within a given period. This alert will monitor Daily and Weekly periods.


There are tow types of this script "HDFS Storage Capacity Usage (Daily)" and "HDFS Storage Capacity Usage (Weekely)"


In your screenshot we see it it "HDFS Storage Capacity Usage (Weekely)"

625604664B means almost 625 MB
2992430585B means almost 3000 MB
598486117B which is almost 598 MB  This is average.


By default the "HDFS Storage Capacity Usage (Weekly)" has the following limits set:

Growth Rate CRITICAL: 20%
Growth Rate WARNING: 10% 


20% of 3000 MB is = 600 MB (Where as your HDFS growth in a week is 625 which is slightly more than 20% .. about 21%) Hence you see this CRITICAL message.

.

This alert does not have to do anything with the Total Storage Size Or Available Storage Size.

This alert is mainly concerned about the Weekely Usage Growth Rate of the Storage. You can increase the default "Growth Rate CRITICAL: 20%" if you see that 20% weekely growth in Storage Usage Weekely is normal in your scenario.

avatar

@Jay SenSharma

Thank you for your detailed explanation.