Created 11-22-2016 02:47 AM
Hi All,
I have two physical servers, each sever have 8 core and 128G RAM.
I plan deploy HDP 2.5 cluster on those server.
When I build cube on Slave2, the performace very bad(High CPU, RAM). But when I check performance on Master and Slave1, I see it free a lot of RAM and CPU.
So is it ok when I take the resource from Master and Slave1 to Slave2. Because In my cluster, only Slave2 have high workload.
Or can you advice my other better design for me.
Thanks,
Created 11-22-2016 03:40 AM
A two node cluster is never a great idea. co locating masters with with slaves is ok when workload is low for POC or simply dev testing. since you are using kylin, it reads hive table, builds cubes and stores in hbase. That activity is very CPU and IO bound. I would say lower your workload throughput (lower volume of data) or add more nodes.
Created 11-22-2016 03:53 AM
So I need running 2 masters nodes with 1 slaves node, and plan to add more node in future, that right ?.
If I install ESXi in my physical server and spare to multi small nodes(16G Ram each) . Is the performance better than install on big node( physical 64G RAM) ?
If do that I will have a lot of node running, but the RAM, CPU must be depend on physical servers.
Created 11-22-2016 03:56 AM
with 1 slaves node that is only 1 region server. for hbase that is no good. hbase you need min 3 nodes. now if you limit your volumes you can get away with it. just don't expect great performance. all your workload is going to single RS