Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka mirror maler deployment

Highlighted

Kafka mirror maler deployment

Explorer

I have a Kafka cluster composed by 5 brokers and 4 mirror maker to mirror date from 2 different data centers. I know that a kafka broker requires its own dedicated hardware especially because of the high disk I/O, memory usage and CPU intensive application.

I would like to know if could make sense to deploy a mirror maker process on a node that is even a Kafka broker or if I should consider to have the mirror maker on:

- a dedicated node

- a node which hostes a zookeeper server

 

HDFS and others cloudera services are deployed on different nodes.

Thanks in advance,

Beniamino

1 REPLY 1

Re: Kafka mirror maler deployment

Master Guru
Since it does not do intensive compute (just relays data as-is), it should be OK to run it on a Broker host.

You can setup a trigger to monitor its CPU/Memory use so over time you can get alerted on if its actually beginning to demand higher resources. Until that happens, its OK to keep it on the Broker itself.