question Re: What is the Best consolidated Guide for HBase Tuning? in Archives of Support Questions (Read Only)

What is the Best consolidated Guide for HBase Tuning?

wfloyd — Sun, 11 Oct 2015 19:15:53 GMT

In your opinion, what is the best consolidated resource for performance tuning an HBase instance?

Are the HBase guides the best reference we have for performance tuning or are there others which are more comprehensive?

Re: What is the Best consolidated Guide for HBase Tuning?

nmaillard1 — Sun, 11 Oct 2015 19:28:13 GMT

Hey Wes

These are a good start. The Hbase SME wiki has a couple of links as well for Hbase tuning and staffing. this link also has info: http://fr.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok

Then there are element specific thoughts like cache algos, number versions, number hfiles etc.. that can vary depending in your usage.

Re: What is the Best consolidated Guide for HBase Tuning?

jstraub — Mon, 12 Oct 2015 21:17:05 GMT

Hey Wes,

here are some of my notes about HBase (Tuning).

Validate your data model (key design, naming, no. versions, ...)
Check configuration (RAM, BlockSize,...)
Caching
- http://www.n10k.com/blog/blockcache-101/
- http://www.n10k.com/blog/blockcache-showdown/

Compression (Gzip,Snappy,...)
How is data retrieved?

Random lookups => Maybe smaller block size makes more sense?
Sequential scans => Maybe higher block size makes more sense?

Check whether you have hot spots in your Hbase Environment?

http://blog.sematext.com/2012/04/09/hbasewd-avoid-regionserver-hotspotting-despite-writing-records-with-sequential-keys/
"Hot Spotting is an hbase phenomenon where a region server is hosting most sought after data. This causes that region server to run really hot and potentially slow down and run in a degraded mode."

Metrics to check

BlockCache Ratio and Hit/Miss
Compaction Queue
Memstore size
Flush Queue
Call Queue
CPU load & WIO
Memory usage
Latency
IOPS
JVM Metrics (GC, Log,...)

Links:

Not a consolidated guide, but it might help anyway 🙂

Let me know if you need more information about metrics or specific tuning methods (RAM, blockSize, etc.) I might have some more documents on my drive.

Jonas