question Re: Watch region merge progress in HBase in Archives of Support Questions (Read Only)

Watch region merge progress in HBase

l_mourikis — Fri, 02 Nov 2018 10:39:08 GMT

Hello,

I need to watch the progress of a region merge task in HBase 1.2 . I need it to use it on a script, so I would prefer a solution that works best through linux shell and not Web UI.

Regards

Re: Watch region merge progress in HBase

elserj — Fri, 02 Nov 2018 21:05:42 GMT

Region merging is an asynchronous process in HBase.

Your only option is to read and parse the relevant data in hbase:meta, and use this to determine when both daughter Regions that you requested to be merged together are combined into a single parent Region.

Re: Watch region merge progress in HBase

l_mourikis — Fri, 02 Nov 2018 21:19:43 GMT

Thank you Josh!

Exactly, merging is asynchronous indeed, thus the need to monitor when it's completed, so that I know when Ican submit the next merge request without overloading the system.

I saw there is the command list_procedures. Do you think this can be used to monitor any non-completed merge processes?

If not, is there an easy way to parse the hbase:meta through shell to obtain the required info? What should I look for? If I am getting it right, I should see all three regions while merge is active (one new + two old) and just the new one when it will have completed. So, when the two old regions are no longer present, merge can be considered completed. Does that sounds correct? Of course, I would prefer to just use a simpler command, like list_processes, if it is applicable.

Regards

Re: Watch region merge progress in HBase

elserj — Fri, 02 Nov 2018 21:31:20 GMT

Easy way to parse meta? Probably not :). At least, not "easy" in comparison to what you'd expect from our public API. It's definitely doable, but will require a little digging on your part.

Not sure the exact version of HBase you're using. You may have some luck with list_procedures, but hard to say off the cuff.

Re: Watch region merge progress in HBase

l_mourikis — Fri, 02 Nov 2018 21:44:20 GMT

Thanks Josh! I will give it a try in the next couple of days (need to build a test environment first) and return with results.

Re: Watch region merge progress in HBase

l_mourikis — Tue, 13 Nov 2018 22:27:44 GMT

In the end it was a lot easier. I am monitoring the existence of the original regions right after the merge_region command. When both regions are gone, this can be considered as a sign that the merge is complete. Of course the regions can disappear due to another reason (maybe an automatic split) but for the specific task it covers my needs.