Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka mirror maler deployment

Kafka mirror maler deployment

Explorer

I have a Kafka cluster composed by 5 brokers and 4 mirror maker to mirror date from 2 different data centers. I know that a kafka broker requires its own dedicated hardware especially because of the high disk I/O, memory usage and CPU intensive application.

I would like to know if could make sense to deploy a mirror maker process on a node that is even a Kafka broker or if I should consider to have the mirror maker on:

- a dedicated node

- a node which hostes a zookeeper server

 

HDFS and others cloudera services are deployed on different nodes.

Thanks in advance,

Beniamino

1 REPLY 1
Highlighted

Re: Kafka mirror maler deployment

Master Guru
Since it does not do intensive compute (just relays data as-is), it should be OK to run it on a Broker host.

You can setup a trigger to monitor its CPU/Memory use so over time you can get alerted on if its actually beginning to demand higher resources. Until that happens, its OK to keep it on the Broker itself.