Rate of growth HDFS Filesystem

[#1]

I need to know the rate of growth of my HDFS file system for past 6 months. I did browse Ambari and I do not see in Name Node UI too.

Is there any other UI I need to check ?

Or Is there any hdfs admin commnds to get this information ?

[#2]

@Kumar Veerapan Im not sure whether I got your question right. But If you are asking for UI to check the rate of file growth then do check this https://www.datadoghq.com/product/

Also grafana https://grafana.com/grafana

Helped me a lot in terms of time series metrics.

[#3 · Accepted solution]

@Kumar Veerappan

Ambari provides some inbuilt alerts to findout the Weekly/Daily growth in the HDFS usage.

Ambari UI --> Alerts (Tab) --> "Alert Definition Filter" Search for "HDFS Storage Capacity Usage"

This service-level alert is triggered if the increase in storage capacity usage deviation has grown beyond the specified threshold within a given period. This alert will monitor Daily and Weekly periods.

Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/hdfs_service_a...

However if you want to get this data for 6 months then you might have to write your own custom alert script.

\Sometime back i have written a basic example of how we can have our own custom ambari alert. https://community.hortonworks.com/articles/38149/how-to-create-and-register-custom-ambari-alerts.htm...

Grafana basically fetches data from AMS (Ambari Metrics Collector) using APIs so the data need to be available on AMS first.