Created 09-13-2017 06:03 AM
I want to trigger major compaction for all the tables in the hbase. Using hbase client API, major compaction is triggered for all the times via admin.majorCompact(tableName).
1) How to figure out the completion status for the compaction, since hbase client api majorCompact is an Asynchorous process.2) Is it mandatory to wait until compaction process completion , to query hbase for real time process
Created 09-13-2017 06:15 AM
bq. 1) How to figure out the completion status for the compaction, since hbase client api majorCompact is an Asynchorous process
you can use below API.
CompactionState compactionState = admin.getCompactionState(table.getName());
2) Is it mandatory to wait until compaction process completion , to query hbase for real time process
Check your resources consumption with compaction as it impacts I/O, CPU usage and network . In standard server configuration , it is fine to run real time process during compaction.
Created 09-13-2017 07:12 AM
Hey Ankit, thanks for your response.
Hbase Client returns NONE though the compaction is completed for the table(which can be identifed from the logs).
And major compaction is an asychronous process, how to predict the wait time to get the state form the client API (In case if it returns the state properly)
Created 09-13-2017 07:18 AM
CompactionState.NONE : means no compaction is currently running.
bq. And major compaction is an asychronous process, how to predict the wait time to get the state form the client API
You need to keep on polling the API with some arbitrary wait time.
Created 09-13-2017 08:26 AM
After polling for some time, if the state returns MAJOR, shall we confirm major compaction is completed?.
admin.majorCompact(TableName.valueOf("testTable"));
Thread.sleep("sometime");
CompactionState state=admin.getCompactionState("testTable");
if(state.equals(CompactionState.MAJOR))
{ //Major compaction is completed }
This code always returns state as NONE
Created 09-13-2017 08:29 AM
NONE means no compaction is currently running (or completed) , if state return MAJOR , it means major compaction is still running. Either your table is small or you have a single file in each region resulting in major compaction to complete soon and showing state as NONE
Created 09-13-2017 06:23 PM
bq. Thread.sleep("sometime");
How long was the sleep ?
I might be possible that compaction finished after the sleep.
Which HDP version are you using ?
Created 09-14-2017 01:20 PM
Hadoop: 2.7.3 , HBase 1.3.1
Major compaction has been triggered for all the tables in hbase via hbase client.
for(String table: listofTables)
admin.majorCompact(table)
Compaction has been triggered for all the tables which can be confirmed from hbase logs, but completion logs are not present for all the tables(checked logs even after two hours of compaction trigger)