Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

3-nodes Cluster Optimization for HBase / OpenTSDB

Highlighted

3-nodes Cluster Optimization for HBase / OpenTSDB

New Contributor

Hi

I am planning to create small 3-nodes cluster for OpenTSDB:

- HDFS NameNode: node1

- HDFS SNameNode: node2

- HDFS DataNode: all nodes

- HBase Master: node1

- HBase RegionServer: all nodes

- Zookeepeer: all nodes

- OpenTSDB: all nodes

- Cloudera Manager: node3

Node Spec: 8+ core, 32 or 64 GB RAM, RAID10

Please help me to find recommended Java Heap Sizes for:

- HBase Master

- HBase RegionServer

- OpenTSDB

- HDFS NameNode

- HDFS DataNode

- HDFS Balancer

for two hardware spec variants (32 and 64 GB RAM).

Any additional tweaks are appreciated.

Current OpenTSDB database size: 500 GB

Current OpenTSDB Client Connections: 3000+

OpenTSDB Tweaks:

- compression: lzo

- tsd.core.auto_create_metrics = true

- tsd.storage.fix_duplicates = true

- tsd.core.uid.random_metrics = true

- tsd.storage.hbase.prefetch_meta = true

- tsd.storage.salt.width = 1

- tsd.storage.salt.buckets = 3

Other Tweaks:

- THP: disabled

- EXT4: noatime, nodiratime

- vm.swappiness = 0

- net.core.somaxconn = 1024

- vm.min_free_kbytes = 1048576

- vm.zone_reclaim_mode = 0

- hard/soft nofile limit: 65000

Thank you

Don't have an account?
Coming from Hortonworks? Activate your account here