question Re: Confusing metrics for HBase requests from Cloudera Manager in Archives of Support Questions (Read Only)

Confusing metrics for HBase requests from Cloudera Manager

sergey.sheypak566881637 — Fri, 16 Sep 2022 09:29:44 GMT

Hi, hbase-master says that I have 1300-1500 requests per second for the whole cluster

Cloudera Manager 5.2.1 says that I have:

4500 read requests per second

500 write request per second.

I see that graphic on HBase service page.

Who Is lying?

Re: Confusing metrics for HBase requests from Cloudera Manager

GautamG — Tue, 26 May 2015 01:14:50 GMT

Could you please check if either or both the HBase Canary (within HBase)
and HBase Region Health Canary (in Service Monitor) are turned on? I
suspect the Canary reads are what is skewing the stats. How many regions do
you have overall?

Re: Confusing metrics for HBase requests from Cloudera Manager

sergey.sheypak566881637 — Tue, 26 May 2015 08:54:05 GMT

Hi, thanks for the reply

I found these properties in HBase service configuration:

Enable HBase Canary=true

HBase Region Health Canary=true

>(in Service Monitor)

Didn't find there someting related to HBase

Re: Confusing metrics for HBase requests from Cloudera Manager

GautamG — Tue, 02 Jun 2015 12:16:38 GMT

The most likely cause is that the HBase Canary (part of HBase, not from Service Monitor) is pinging each region's health regularly.

Prior to CM5.3.2, turning this HBase Canary off didn't work. So you could either
- upgrade to CM 5.3.2 (or later). This should not lead to cluster downtime as only CM and agents will be upgraded.
- lower the frequency of the the HBase Canary so that the discrepancy in stats is reduced.