- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Rate of growth HDFS Filesystem
- Labels:
-
Apache Hadoop
Created ‎03-31-2017 11:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I need to know the rate of growth of my HDFS file system for past 6 months. I did browse Ambari and I do not see in Name Node UI too.
Is there any other UI I need to check ?
Or Is there any hdfs admin commnds to get this information ?
Created ‎03-31-2017 01:51 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ambari provides some inbuilt alerts to findout the Weekly/Daily growth in the HDFS usage.
Ambari UI --> Alerts (Tab) --> "Alert Definition Filter" Search for "HDFS Storage Capacity Usage"
This service-level alert is triggered if the increase in storage capacity usage deviation has grown beyond the specified threshold within a given period. This alert will monitor Daily and Weekly periods.
Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/hdfs_service_a...
However if you want to get this data for 6 months then you might have to write your own custom alert script.
\Sometime back i have written a basic example of how we can have our own custom ambari alert. https://community.hortonworks.com/articles/38149/how-to-create-and-register-custom-ambari-alerts.htm...
Grafana basically fetches data from AMS (Ambari Metrics Collector) using APIs so the data need to be available on AMS first.
Created ‎03-31-2017 01:32 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Kumar Veerapan Im not sure whether I got your question right. But If you are asking for UI to check the rate of file growth then do check this https://www.datadoghq.com/product/
Also grafana https://grafana.com/grafana
Helped me a lot in terms of time series metrics.
Created ‎03-31-2017 01:51 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ambari provides some inbuilt alerts to findout the Weekly/Daily growth in the HDFS usage.
Ambari UI --> Alerts (Tab) --> "Alert Definition Filter" Search for "HDFS Storage Capacity Usage"
This service-level alert is triggered if the increase in storage capacity usage deviation has grown beyond the specified threshold within a given period. This alert will monitor Daily and Weekly periods.
Please see: https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-user-guide/content/hdfs_service_a...
However if you want to get this data for 6 months then you might have to write your own custom alert script.
\Sometime back i have written a basic example of how we can have our own custom ambari alert. https://community.hortonworks.com/articles/38149/how-to-create-and-register-custom-ambari-alerts.htm...
Grafana basically fetches data from AMS (Ambari Metrics Collector) using APIs so the data need to be available on AMS first.
