Support Questions
Find answers, ask questions, and share your expertise

I am using HDP 2.1.

Hi everyone,

Is there any way to take a output of hdfs disk utilization for last 1 week in HDP 2.1. Means I want to take output of my hdfs disk utilization for last one week from command.

I have following component installed on cluster

6 REPLIES 6

@Ashnee Sharma

Something like similar ambari reports as part of the alerting. You can find the alert with name:

1. HDFS Storage Capacity Usage (Daily)

2. HDFS Storage Capacity Usage (Weekly)

However you may need to check the logic for this alert throeshold calculation.

Also those alerts just watch the growth rates.

@Ashnee Sharma

Also at the HDFS level you may need to explore the MBean:

http://erie1.example.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem

and query: "CapacityUsedGB" and "CapacityTotalGB" on your own on a desired interval to see the growth.

.

Example:

$ curl -s http://erie1.example.com:50070/jmx?qry=Hadoop:service=NameNode,name=FSNamesystem | grep  "Capacity*"

    "CapacityTotal" : 333359300608,
    "CapacityTotalGB" : 310.0,
    "CapacityUsed" : 10163200102,
    "CapacityUsedGB" : 9.0,
    "CapacityRemaining" : 191658755293,
    "CapacityRemainingGB" : 178.0,
    "CapacityUsedNonDFS" : 131537345213,
    "BlockCapacity" : 2097152,

.

@jss

My query is last I want to get the disk utilization form today to last week, means how much my partition is growing everyday.

I mean to say that I want total space utilization on everyday base for last one week.

@Ashnee Sharma

My apology if i misunderstood your query.

However I do not see any standard in-built utility in HDFS which can give you the exact detail (like how much data in a day your HDFS is utilizing and total how much in a week).

I guess you will need to calculate it using the attributes provided by the "Hadoop:service=NameNode,name=FSNamesystem" OR using the su - hdfs -c "hdfs dfsadmin -report" report calculation.

@jss

I can calculate this from today to next week but I need this analysis for past week.