Reply
Highlighted
Explorer
Posts: 10
Registered: ‎01-31-2017

Kafka mirror maler deployment

[ Edited ]

I have a Kafka cluster composed by 5 brokers and 4 mirror maker to mirror date from 2 different data centers. I know that a kafka broker requires its own dedicated hardware especially because of the high disk I/O, memory usage and CPU intensive application.

I would like to know if could make sense to deploy a mirror maker process on a node that is even a Kafka broker or if I should consider to have the mirror maker on:

- a dedicated node

- a node which hostes a zookeeper server

 

HDFS and others cloudera services are deployed on different nodes.

Thanks in advance,

Beniamino

Posts: 1,760
Kudos: 379
Solutions: 282
Registered: ‎07-31-2013

Re: Kafka mirror maler deployment

Since it does not do intensive compute (just relays data as-is), it should be OK to run it on a Broker host.

You can setup a trigger to monitor its CPU/Memory use so over time you can get alerted on if its actually beginning to demand higher resources. Until that happens, its OK to keep it on the Broker itself.
Announcements