Reply
Highlighted
New Contributor
Posts: 4
Registered: ‎08-18-2016

HBase slow while loading parallel using Hive

Hi,

 

We are having performance issue in HBase.

1. We can't perform loading parallel using Hive:

    If we trigger more that one data loading jobs in parallel. Only one is running and others are waiting status. They are completing as soon as first one get completed.

 

2. Sometimes data insert jobs are getting failed due OutOfMemoryError.

...........................................................................................................................................................
export HBASE_MASTER_OPTS=
+ HBASE_MASTER_OPTS=
++ replace_pid -Xms976224256 -Xmx976224256 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/CD-HBASE-kAppOeMj_CD-HBASE-kAppOeMj-REGIONSERVER-06c33626eaa981cdb3f58513a30e2245_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
++ sed 's#{{PID}}#3326#g'
++ echo -Xms976224256 -Xmx976224256 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError '-XX:HeapDumpPath=/tmp/CD-HBASE-kAppOeMj_CD-HBASE-kAppOeMj-REGIONSERVER-06c33626eaa981cdb3f58513a30e2245_pid{{PID}}.hprof' -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh
+ export 'HBASE_REGIONSERVER_OPTS=-Xms976224256 -Xmx976224256 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/CD-HBASE-kAppOeMj_CD-HBASE-kAppOeMj-REGIONSERVER-06c33626eaa981cdb3f58513a30e2245_pid3326.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
+ HBASE_REGIONSERVER_OPTS='-Xms976224256 -Xmx976224256 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/CD-HBASE-kAppOeMj_CD-HBASE-kAppOeMj-REGIONSERVER-06c33626eaa981cdb3f58513a30e2245_pid3326.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh'
..................................................................

Please guide me how to optimise HBase to avoid this issues.

 

Thanks & regards,

Saktipada Maity

Announcements