OK. I had a meandering circuit to get a version of operator-tools built and running.   After running it on the meta table - sudo hbase hbck -j /tmp/hbase-hbck2-1.1.0-SNAPSHOT.jar extraRegionsInMeta alfa:rfilenameext --fix   I have ended up with:-   alfa:extgen,F,1608197544264.ba22f6113a0bb9520cee1f7b30050fa7. column=info:state, timestamp=1608197544904, value=OPEN alfa:rfilenameext column=table:state, timestamp=1604493139455, value=\x08\x00 alfa:rfiles column=table:state, timestamp=1602600776355, value=\x08\x00   I still have one row referring to the missing table.  Do I need to restart hbase service to remove this or will it vanish at some time?    HBASE on the master still shows the 48 regions in transition when I open the interface.  I assume this is because the service has not been restarted.   ... View more

I got this when I ran it:- ERROR: Unsupported command: extraRegionsInMeta   It seems our version of HBASE may not support this command.   So I am left with the only option of trying to manually remove these rows from meta table...and I am a complete newbie in this regard. ... View more

Sorry, I see that the version of the README that is on this page https://github.com/apache/hbase-operator-tools/tree/master/hbase-hbck2 has extraRegionsinMeta extraRegionsInMeta <NAMESPACE|NAMESPACE:TABLENAME>... Options: -f, --fix fix meta by removing all extra regions found. Reports regions present on hbase:meta, but with no related directories on the file system. Needs hbase:meta to be online. For each table name passed as parameter, performs diff between regions available in hbase:meta and region dirs on the given file system. Extra regions would get deleted from Meta if passed the --fix option. NOTE: Before deciding on use the "--fix" option, it's worth check if reported extra regions are overlapping with existing valid regions. If so, then "extraRegionsInMeta --fix" is indeed the optimal solution. Otherwise, "assigns" command is the simpler solution, as it recreates regions dirs in the filesystem, if not existing. An example triggering extra regions report for tables 'table_1' and 'table_2', under default namespace: $ HBCK2 extraRegionsInMeta default:table_1 default:table_2 An example triggering extra regions report for table 'table_1' under default namespace, and for all tables from namespace 'ns1': $ HBCK2 extraRegionsInMeta default:table_1 ns1 Returns list of extra regions for each table passed as parameter, or for each table on namespaces specified as parameter.  but the README that is contained in the .tgz file does NOT have this information and it is misleading to have it included in one and not within the actual src/bin distribution. Command: addFsRegionsMissingInMeta <NAMESPACE|NAMESPACE:TABLENAME>... Options: -d,--force_disable aborts fix for table if disable fails. To be used when some regions may be missing from hbase:meta but their directories are present in HDFS. This is a 'lighter' version of 'OfflineMetaRepair' tool commonly used for similar issues in hbase-1.x. This command needs hbase:meta to be online. For each table name passed as parameter, it performs a diff between regions available in hbase:meta and region dirs on HDFS. Then for dirs with no hbase:meta matches, it reads the 'regioninfo' metadata file and re-creates given region in hbase:meta. Regions are re-created in 'CLOSED' state in the hbase:meta table, but not in the Masters' cache, and they are not assigned either. To get these regions online, run the HBCK2 'assigns'command printed when this command-run completes. NOTE: If using hbase releases older than 2.3.0, a rolling restart of HMasters is needed prior to executing the provided 'assigns' command. An example adding missing regions for tables 'tbl_1' in the default namespace, 'tbl_2' in namespace 'n1' and for all tables from namespace 'n2': $ HBCK2 addFsRegionsMissingInMeta default:tbl_1 n1:tbl_2 n2 Returns HBCK2 an 'assigns' command with all re-inserted regions. SEE ALSO: reportMissingRegionsInMeta assigns [OPTIONS] <ENCODED_REGIONNAME>... Options: -o,--override override ownership by another procedure A 'raw' assign that can be used even during Master initialization (if the -skip flag is specified). Skirts Coprocessors. Pass one or more encoded region names. 1588230740 is the hard-coded name for the hbase:meta region and de00010733901a05f5a2a3a382e27dd4 is an example of what a user-space encoded region name looks like. For example: $ HBCK2 assign 1588230740 de00010733901a05f5a2a3a382e27dd4 Returns the pid(s) of the created AssignProcedure(s) or -1 if none. bypass [OPTIONS] <PID>... Options: -o,--override override if procedure is running/stuck -r,--recursive bypass parent and its children. SLOW! EXPENSIVE! -w,--lockWait milliseconds to wait before giving up; default=1 Pass one (or more) procedure 'pid's to skip to procedure finish. Parent of bypassed procedure will also be skipped to the finish. Entities will be left in an inconsistent state and will require manual fixup. May need Master restart to clear locks still held. Bypass fails if procedure has children. Add 'recursive' if all you have is a parent pid to finish parent and children. This is SLOW, and dangerous so use selectively. Does not always work. filesystem [OPTIONS] [<TABLENAME>...] Options: -f, --fix sideline corrupt hfiles, bad links, and references. Report on corrupt hfiles, references, broken links, and integrity. Pass '--fix' to sideline corrupt files and links. '--fix' does NOT fix integrity issues; i.e. 'holes' or 'orphan' regions. Pass one or more tablenames to narrow checkup. Default checks all tables and restores 'hbase.version' if missing. Interacts with the filesystem only! Modified regions need to be reopened to pick-up changes. fixMeta   Regardless, if it does exist then it should do the job I want.  I shall reveal the outcome after execution.     ... View more

I downloaded and extracted hbase-operator-tools.1.0.0 via link.   Downloads\hbase-operator-tools-1.0.0-bin.tar\hbase-operator-tools-1.0.0\hbase-hbck2   The readme has no mention of 'extraregionsinmeta' so I am a bit confused as to where that comes from.   By the way, not sure if this is a factor but we have active and standby masters.   HBase Version 2.1.0-cdh6.3.2 ... View more

Getting it and/or building it is not the problem. Does it have the facility to remove redundant meta information, and without corrupting other meta data for needed tables?   Why is 'delete' or 'deleteall' unsuitable, is there no way to specify removal of rows containing that table name?  After all, if it is just a table like every other table then there must exist a normal database method to selectively remove data.   Is it possible to eradicate transition errors (in this case) by manually altering the state to OPENING -> OPEN -> CLOSING -> CLOSED... would this 'finalise' transition errors to the point they get removed from meta by regular hbase operation? ... View more

Hi, @smdas,   In my post I neglected to mention that (last week) I shut hbase, removed the masterprocwals, dfs removed the structure, zk'd the znodes, and restarted hbase.  This removed all references to the affected table.  There are (currently) no locks or procedures for this table.   The hbase UI reports those regions in transition - I assume only because they are contained in hbase:meta - but there is nothing in storage, nothing in zk, and CM shows no table details as that table is not there.   I have been poring over info regarding use of hbck2 and cannot seem to find anything resembling a suitable operation that will actually operate on hbase:meta rows, just general options for fixing meta as a kind of global operation.     ... View more