Support Questions

Find answers, ask questions, and share your expertise

HBase: How to remove corrupted hbase:meta rowkeys containing single commas

avatar
New Contributor

Hi all,

I've run into an interesting issue where corrupted entries with rowkeys containing a single comma (,) were inserted into hbase:meta. These rows show up in scans, but get, delete, and deleteall operations don't work on them. Example of such a row:

ROW COLUMN+CELL
rowkey, column=info:regioninfo, timestamp=..., value=...

Attempts to remove them with the Java client and HBCK2 (fixMeta) didn't work.

Has anyone dealt with this before? Is there a safe way to remove or repair such dangling/corrupted rowkeys from hbase:meta?

In addition, I have opened this upstream issue reporting the problem: https://issues.apache.org/jira/browse/HBASE-29554

Thanks in advance for any guidance.

4 REPLIES 4

avatar
Community Manager

@EmilKle Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our HBase experts @smdas @blizano @pajoshi  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
New Contributor

@DianaTorres thank you for your prompt reply.
I haven't found a solution to this problem as of now. If you have any suggestions on how to solve this issue and get rid of the corrupted meta entries in the existing cluster, I would highly appreciate it.

Thank you in advance.

avatar
Community Manager

Hi @shubham_sharma @ale241290xdf @aquilodran @David666 Do you have any insights here? Thanks!


Regards,

Diana Torres,
Senior Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
New Contributor

Just to add even more input on this, I can see a side effect of it shown in the HBCK report under Inconsistent Regions 

Just to add more input on this, in the HBase master logs I can also see:

<2025-09-16T10:11:27.734+0200> <WARN> <janitor.ReportMakingVisitor>: <INCONSISTENCY: Row name is not equal to serialized info:regioninfo content; row=XXX:XXX,e14ef354e1644a6a2b229ffde05f38d3 XXX:XXX,test,12345.07a983faf71bc641bfbb4e8736d378ac.; See if RegionInfo is referenced in another hbase:meta row? Delete?>

In meta:

XXX:XXX,e14ef354e1644a6a2b229ffde05f38d3 column=info:regioninfo, timestamp=2025-08-26T12:12:21.389, value={ENCODED => 07a983faf71bc641bfbb4e8736d378ac, NAME => 'XXX:XXX,test,12345.07a983faf71bc641bfbb4e8736d378ac.', STARTKEY => 'test', ENDKEY => ''}