Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hbase 2.0 table stuck in 'DISABLING' state and region is stuck in 'OPENING' state

avatar
Explorer

I have a table stuck in 'DISABLING' state and one of the four table regions is stuck in 'OPENING' state. Hbase 2.0 hbck doesn't allow fixing tables.

./hbase zkcli, ls /hbase-unsecure/table/<table name> doesn't show anything

restarting the hbase master, regionservers, and zookeeper doesn't resolve

How do I fix this issue.

1 ACCEPTED SOLUTION

avatar
Explorer

This was a temp table. I wasn't able to drop the table b/c it was in a 'DISABLING' state, not 'DISABLED'. I was able to figure out a way to manually remove the table.

Remove the table directory in hdfs, /apps/hbase/data/data/default/<table_name>.

Remove the rows in hbase:meta that referenced the table, deleteall 'hbase:meta', '<row_name>'

Stop hbase, zookeeper. Start zookeeper, hbase.

View solution in original post

5 REPLIES 5

avatar
Explorer
hbase(main):002:0> is_enabled 'EDA_CONTRACTS_TEMP'
false
Took 0.1362 seconds
=> false
hbase(main):003:0> is_disabled 'EDA_CONTRACTS_TEMP'
false
Took 0.0290 seconds
=> 1

./hbase hbck -details
Number of regions in transition: 1
  EDA_CONTRACTS_TEMP,,1545342660748.52fdc258feeeae9285551f2cc231d841. state=OPENING, ts=Thu Jan 10 19:07:32 MST 2019 (177746s ago), server=null


avatar
Explorer

When I try to unassign the region (close) I see this in the log. Is there a way to manually set the state of the region or force it to close.

2019-01-13 12:46:06,272 WARN [PEWorker-16] assignment.RegionTransitionProcedure: Failed transition, suspend 2048secs pid=2931, state=RUNNABLE:REGION_TRANSITION_DISPATCH; UnassignProcedure table=EDA_CONTRACTS_TEMP, region=52fdc258feeeae9285551f2cc231d841, server=hadoop-2,16020,1546877712972; rit=OPENING, location=hadoop-2,16020,1546877712972; waiting on rectified condition fixed by other Procedure or operator intervention org.apache.hadoop.hbase.exceptions.UnexpectedStateException: Expected [SPLITTING, SPLIT, MERGING, OPEN, CLOSING] so could move to CLOSING but current state=OPENING

avatar
Explorer

This was a temp table. I wasn't able to drop the table b/c it was in a 'DISABLING' state, not 'DISABLED'. I was able to figure out a way to manually remove the table.

Remove the table directory in hdfs, /apps/hbase/data/data/default/<table_name>.

Remove the rows in hbase:meta that referenced the table, deleteall 'hbase:meta', '<row_name>'

Stop hbase, zookeeper. Start zookeeper, hbase.

avatar
Explorer

This was a temp table. I wasn't able to drop the table b/c it was in a 'DISABLING' state, not 'DISABLED'. I was able to figure out a way to manually remove the table.

Remove the table directory in hdfs, /apps/hbase/data/data/default/<table_name>.

Remove the rows in hbase:meta that referenced the table, deleteall 'hbase:meta', '<row_name>'

Stop hbase, zookeeper. Start zookeeper, hbase.

avatar
Rising Star

Hi,


No need to do any deletion. Just follow the steps here to resolve the issue without any data loss: https://community.hortonworks.com/questions/242343/hbase-table-is-stuck-in-disabling-state-neither-e...


Thanks,

Shesh