Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Too frequent major compaction

Too frequent major compaction

Contributor

I have set major compaction frequency to 3 days.

	hbase.offpeak.start.hour = 19
	hbase.offpeak.end.hour = 1

However I see in RS metrics, for a particular table number of total major compaction is 22 ("Namespace_default_table_user_states_metric_majorCompactionTime_num_ops": 22) and the cluster was last restarted 14 days ago. So this is >1 major compaction a day.

Also in logs I see many major compaction running on single file. Despite it's set for 3 to 10 files. ( see the first line below)

2017-06-08 02:32:07,377 INFO  [StoreOpener-1f254a063af7b15509846ff27ef9b5a8-1] compactions.CompactionConfiguration: size [134217728, 9223372036854775807); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 259200000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, base window in milliseconds 21600000, windows per tier 4,incoming window min 6


2017-06-08 00:28:36,581 INFO  [regionserver/<host>/10.10.210.195:16020-shortCompactions-1495621699452] regionserver.HStore: Completed major compaction of 1 (all) file(s) in f of events,voodoo.com:ringo:call_log:1433665626397:r116qjoprsjhppq1,1435904225768.c901139ea575133398f86ceb4dc253fa. into f275a4737fe1485e9058532a2226d401(size=13.7 M), total size for store is 13.7 M. This selection was in queue for 0sec, and took 2sec to execute.2017-06-08 

06:02:22,176 INFO  [regionserver/<host>/10.10.210.195:16020-shortCompactions-1495621699452] regionserver.HStore: Completed major compaction of 1 (all) file(s) in f of events,voodoo.com:ringo:call_log:1458694092743:7azkyvazxry7rxkr,1461000798058.5703747043306593388ed83c0d245772. into 168f131831244c9f8aafdcc9105e37dc(size=304.3 M), total size for store is 304.3 M. This selection was in queue for 0sec, and took 27sec to execute.

2017-06-08 06:03:55,214 INFO  [regionserver/<host>/10.10.210.195:16020-shortCompactions-1495621699452] regionserver.HStore: Completed major compaction of 1 (all) file(s) in f of events,voodoo.com:ringo:connectivity_log:1464497220532:y66a0afayoava6gn,1466007331891.d89b88bd976aab7e229116fcf4ac2fb0. into e7f6451d5f2a40a59795285c2d5c8feb(size=1.0 G), total size for store is 1.0 G. This selection was in queue for 28sec, and took 1mins, 32sec to execute.

This log line I find several times in a day and it's not limited to off-peak hours. I do not think frequency of major compaction should be this high. Or maybe the log line is suggesting something else and not major compaction.

hbase.hregion.max.filesize is 20G

hbase.hstore.compaction.max is 10

Any help with figuring out what is happening is appreciated. Please let me know if any more information is required.

Thank you!

1 REPLY 1
Highlighted

Re: Too frequent major compaction

Contributor

Hi Sanket,


Did you solve this problem?

I am encountring the same problem.. many Major compaction the same day in the same Column Family and the same server?


Thanke

Helmi KHALIFA

Don't have an account?
Coming from Hortonworks? Activate your account here