Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

2 clusters of zookeeper servers

Highlighted

2 clusters of zookeeper servers



We have Hadoop cluster with the following details

3 kafka machines

3 zookeeper servers

Now the Hadoop cluster include also datanode machines And all application are using the zookeeper servers include the kafka machines

Now

We want to do the following changes

We want to add additional 3 zookeeper servers that will be in a separate cluster

And only kafka machine will use this additional zookeeper servers

Is it possible ?

Michael-Bronson
3 REPLIES 3

Re: 2 clusters of zookeeper servers

Community Manager

The above question were originally posted in the Community Help track. On Wed Jun 26 21:14 UTC 2019, a member of the HCC moderation staff moved it to the Design & Architecture track. The Community Help Track is intended for questions about using the HCC site itself, not technical questions about Hadoop clusters.

Re: 2 clusters of zookeeper servers

Super Mentor

@Michael Bronson

Is there any specific reason for having 3 separate ZK servers which belongs to a separate Cluster for Kafka ?

1. We want to add additional 3 zookeeper servers that will be in a separate cluster
2. And only kafka machine will use this additional zookeeper servers


If you are using Ambari Managed Cluster then it will not be possible via Ambari. As it will require additional efforts to explicitly update the Kafka Zookeeper settings. With this approach there will be many other issues like Ambari will notbe able to keep track of the Zookeepers running as part of a Different cluster.

So that does not look like a good idea.

Instead you can think of making the existing ZK servers more robust / high available to keep your data.


A similar discussion can be found on another HCC thread: https://community.hortonworks.com/questions/248413/should-zookeeper-be-run-on-independent-machines-i...


How to decide, how many zookeepers should I have?

https://community.hortonworks.com/questions/35287/how-to-decide-how-many-zookeepers-should-i-have.ht...

Re: 2 clusters of zookeeper servers

@dear Jay about the additional zookeeper - I dont need the ambari management here , the reason that we want additional 3 machines of zookeeper - is because we are faced many problems in kafka , and we think part of the problem are because the current zookeeper server are installed on masters machines and masters machines are include many other services , so we want to add additional 3 zookeeper servers that will serve only the kafka machines , and the current zookeeper swerves will serv all other application , so lets summary the additional 3 zoo servers will only talk with the 3 kafka machines , and the additional 3 zoo servers will not manage under ambari , we know this but the fact we have so many problems in kafka give the idea to add additional 3 zoo servers only for the kafka , so please give your opinion about my suggestion , your opinion is so important because adding new machine are costs money and we not want to do wrong decision

Michael-Bronson