Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Is there a way to find the completion of major compaction completion status via hbase client API?

Is there a way to find the completion of major compaction completion status via hbase client API?

New Contributor

I want to trigger major compaction for all the tables in the hbase. Using hbase client API, major compaction is triggered for all the times via admin.majorCompact(tableName).
1) How to figure out the completion status for the compaction, since hbase client api majorCompact is an Asynchorous process.2) Is it mandatory to wait until compaction process completion , to query hbase for real time process

7 REPLIES 7

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

bq. 1) How to figure out the completion status for the compaction, since hbase client api majorCompact is an Asynchorous process

you can use below API.

CompactionState compactionState = admin.getCompactionState(table.getName());

2) Is it mandatory to wait until compaction process completion , to query hbase for real time process

Check your resources consumption with compaction as it impacts I/O, CPU usage and network . In standard server configuration , it is fine to run real time process during compaction.

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

New Contributor

Hey Ankit, thanks for your response.
Hbase Client returns NONE though the compaction is completed for the table(which can be identifed from the logs).
And major compaction is an asychronous process, how to predict the wait time to get the state form the client API (In case if it returns the state properly)

Highlighted

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

CompactionState.NONE : means no compaction is currently running.

bq. And major compaction is an asychronous process, how to predict the wait time to get the state form the client API

You need to keep on polling the API with some arbitrary wait time.

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

New Contributor

After polling for some time, if the state returns MAJOR, shall we confirm major compaction is completed?.

admin.majorCompact(TableName.valueOf("testTable"));

Thread.sleep("sometime");

CompactionState state=admin.getCompactionState("testTable");

if(state.equals(CompactionState.MAJOR))

{ //Major compaction is completed }

This code always returns state as NONE

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

NONE means no compaction is currently running (or completed) , if state return MAJOR , it means major compaction is still running. Either your table is small or you have a single file in each region resulting in major compaction to complete soon and showing state as NONE

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

Super Collaborator

bq. Thread.sleep("sometime");

How long was the sleep ?

I might be possible that compaction finished after the sleep.

Which HDP version are you using ?

Re: Is there a way to find the completion of major compaction completion status via hbase client API?

New Contributor

Hadoop: 2.7.3 , HBase 1.3.1
Major compaction has been triggered for all the tables in hbase via hbase client.

for(String table: listofTables)
admin.majorCompact(table)

Compaction has been triggered for all the tables which can be confirmed from hbase logs, but completion logs are not present for all the tables(checked logs even after two hours of compaction trigger)