We do our monitoring using OpenTSDB which runs against HBase. Our current cluster is on CDH 5.9 ( Ubuntu 14). Our new cluster is on CDH 5.16.2 (Ubuntu 16). Both are on the same HBase version (1.2.0)
Using a dummy wordcount table which contains about 1700 rows, I used snapshot on the source cluster,
ExportSnapshot to copy the data over and then ran restore_snapshot on the new cluster. This works fine.
When I do the same operation using the meta table which is about 2.2 Gb, it does not work. I see that the regions are getting created but there is no data. Count and scan does not work. HBase Master goes to bad health with the error "regions in transition state over threshold". I tried this multiple times with other smaller tables too. The only way to bring back the HBase master to good health is to disable and drop the table.
Has anyone encountered this issue? Any tips will be highly appreciated.
... View more