Posts: 10
Registered: ‎01-31-2017

Kafka mirror maler deployment

[ Edited ]

I have a Kafka cluster composed by 5 brokers and 4 mirror maker to mirror date from 2 different data centers. I know that a kafka broker requires its own dedicated hardware especially because of the high disk I/O, memory usage and CPU intensive application.

I would like to know if could make sense to deploy a mirror maker process on a node that is even a Kafka broker or if I should consider to have the mirror maker on:

- a dedicated node

- a node which hostes a zookeeper server


HDFS and others cloudera services are deployed on different nodes.

Thanks in advance,


Posts: 1,886
Kudos: 425
Solutions: 300
Registered: ‎07-31-2013

Re: Kafka mirror maler deployment

Since it does not do intensive compute (just relays data as-is), it should be OK to run it on a Broker host.

You can setup a trigger to monitor its CPU/Memory use so over time you can get alerted on if its actually beginning to demand higher resources. Until that happens, its OK to keep it on the Broker itself.