Zookeeper in Metron

[#1]

Following George Vetticaden's article Apache Metron TP1 Deep Dive - the chart depicts how Apache Metron would look on a 10 node cluster. Where are the Zookeeper nodes (I'd assume we'd want 3 to achieve quorum?) Does ZS in Node 2 stand for Zookeeper Server (if so only one instance of ZK?)

[#2]

Are we to assume the Supervisor contains Zookeeper?

[#3]

In a production cluster, absolutely a minimim of three Zookeepers. However in the Metron TP1 as it is only a test environment there is only a single Zookeeper deployed on Node 2.

[#4 · Accepted solution]

@Ryan Cicak good question. @drussell is correct. For Metron TP1, to prevent from using another m4.xlarge ec2 instance and give more resources to the other services, we chose only to use 1 zookeeper. But in production, we would have a minimum of at least 3 for the quorum. Given that we are going to be using zookeeper for other services managing metron's own configs (enrichment config, theat intel config, etc..) and in the future support for SOLR will require Zookeeper, possibly more than 3 will be required.