HBase Region / HFile blocks pinning in HDP

[#1]

• Consider increasing network capacity to overcome the challenge caused due to non locality of blocks.

• Create configuration groups of datanodes exclusively for HBASE, disabling HDFS balancer on this group and allow only hbase balancer. Follow this url Host_Config_Groups to create host config groups.

Few temporary workarounds can also be applied if problem is severe and need immediate attention :-

• Disable HDFS balancer permanently on the cluster and run it manually on need basis. (Please spin a support case and have the situation discussed before implementing this workaround.)

• In case the performance issue needs to be fixed post running of HDFS Balancer, a major compaction could be manually initiated. For performance gains, major compaction is run on off peak hours such as weekends. This article Compaction_Best_Practices is a recommended read here.

• Scheduling major compaction after scheduled balancer rather than vice versa.

• HDFS although has introduced concept of "favored nodes" feature but HBase APIs are not yet equipped to choose specific nodes during data writing.

• Please note that these are expert level configurations and procedures, if unsure of their implications, its always recommended to open a support case with us.

• Refer following Apache URLs to track the progress of region blocks pinning implementation.

https://issues.apache.org/jira/browse/HBASE-13021

https://issues.apache.org/jira/browse/HDFS-6133