What is the Best consolidated Guide for HBase Tuning?

[#1]

In your opinion, what is the best consolidated resource for performance tuning an HBase instance?

Are the HBase guides the best reference we have for performance tuning or are there others which are more comprehensive?

• http://hbase.apache.org/book.html#performance
• http://hbase.apache.org/book.html#important_configurations

[#2]

Hey Wes

These are a good start. The Hbase SME wiki has a couple of links as well for Hbase tuning and staffing. this link also has info: http://fr.slideshare.net/lhofhansl/h-base-tuninghbasecon2015ok

Then there are element specific thoughts like cache algos, number versions, number hfiles etc.. that can vary depending in your usage.

[#3 · Accepted solution]

Hey Wes,

here are some of my notes about HBase (Tuning).

• Validate your data model (key design, naming, no. versions, ...)
• Check configuration (RAM, BlockSize,...)
• Caching
  • http://www.n10k.com/blog/blockcache-101/
  • http://www.n10k.com/blog/blockcache-showdown/

• Compression (Gzip,Snappy,...)
• How is data retrieved?

• Random lookups => Maybe smaller block size makes more sense?
• Sequential scans => Maybe higher block size makes more sense?

Check whether you have hot spots in your Hbase Environment?

• http://blog.sematext.com/2012/04/09/hbasewd-avoid-regionserver-hotspotting-despite-writing-records-w...
• "Hot Spotting is an hbase phenomenon where a region server is hosting most sought after data. This causes that region server to run really hot and potentially slow down and run in a degraded mode."

Metrics to check

• BlockCache Ratio and Hit/Miss
• Compaction Queue
• Memstore size
• Flush Queue
• Call Queue
• CPU load & WIO
• Memory usage
• Latency
• IOPS
• JVM Metrics (GC, Log,...)

Links:

• http://blog.sematext.com/2012/07/16/hbase-memstore-what-you-should-know/
• http://hortonworks.com/blog/introduction-to-hbase-mean-time-to-recover-mttr/
• http://hadoop-hbase.blogspot.de/2014/03/hbase-gc-tuning-observations.html
• http://hadoop-hbase.blogspot.de/2015/01/more-hbase-gc-tuning.html

Not a consolidated guide, but it might help anyway 🙂

Let me know if you need more information about metrics or specific tuning methods (RAM, blockSize, etc.) I might have some more documents on my drive.

Jonas