Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to fetch last rebalance timestamp for hortonwokrs cluster?

Highlighted

how to fetch last rebalance timestamp for hortonwokrs cluster?

New Contributor

i am a newbie to Hadoop and bigdata, while working on my project I have to fetch last rebalance timestamp for the hortonworks cluster.can anyone help me with a solution?

4 REPLIES 4

Re: how to fetch last rebalance timestamp for hortonwokrs cluster?

Super Mentor

@Ujjwal Bose

If the "Rebalance HDFS" operation was started from Ambari UI /(or using Ambari API) calls then you can look at the "/var/log/ambari-server/ambari-server.log" to find out the timestamp:
Example:

# grep 'REBALANCEHDFS' /var/log/ambari-server/ambari-server.log 

31 Oct 2017 12:47:08,167  INFO [ambari-client-thread-28] AmbariManagementControllerImpl:3984 - Received action execution request, clusterName=plain_ambari, request=isCommand :true, action :null, command :REBALANCEHDFS, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='HDFS', componentName='NAMENODE', hostNames=[amb25101.example.com]}], exclusive: false, clusterName :plain_ambari

07 Nov 2017 06:41:21,549  INFO [ambari-client-thread-28] AmbariManagementControllerImpl:3984 - Received action execution request, clusterName=plain_ambari, request=isCommand :true, action :null, command :REBALANCEHDFS, inputs :{}, resourceFilters: [RequestResourceFilter{serviceName='HDFS', componentName='NAMENODE', hostNames=[amb25102.example.com]}], exclusive: false, clusterName :plain_ambari

.

Re: how to fetch last rebalance timestamp for hortonwokrs cluster?

New Contributor

@Jay Kumar SenSharma

thanks jay for your reply, but couldn't find anything regarding timestamp in the ambari-server.log...

i am looking for a timestamp(in milliseconds) for the last rebalance of hortonworks cluster, which if i get in JSON or XML format would be better, so that i can parse it for my use.

any related api or command?

Re: how to fetch last rebalance timestamp for hortonwokrs cluster?

Super Mentor

@Ujjwal Bose

After the balancer is started, an output file name where the balancer progress will be recorded is printed on the screen. The administrator can monitor the running of the balancer by reading the output file. The output shows the balancer's status iteration by iteration.

In each iteration it prints the starting time, the iteration number, the total number of bytes that have been moved in the previous iterations, the total number of bytes that are left to move in order for the cluster to be balanced, and the number of bytes that are being moved in this iteration. Normally "Bytes Already Moved" is increasing while "Bytes Left To Move" is decreasing.


But i do not think that the timestamp is recorded in any of the other place.

.

Reference "Balancer.java"

Re: how to fetch last rebalance timestamp for hortonwokrs cluster?

New Contributor

See also: https://community.hortonworks.com/questions/70182/question-on-hdfs-rebalance-1.html?childToView=2368.... The above answer is pretty good, "Need to move / left to move" is a good indicator.