Mirrormaker 2.0 delay and how to identify bottleneck

Question

Hi,I am using mirrormaker 2 to copy data of one topic from source cluster to destination cluster. data is getting copied but I see 3 hours of lag. and I am not able to find the exact cause of it (whether the delay is at consumer side or producer ) as it is using kafka connect framework.Below are the config detailstest topic having 2 partition and replication factor as 3number of messages coming into the source topic is 8K per second.

Babasaheb · Answer

@Onkar_Gagre

I would suggest you to tweak the following settings for better MirrorMaker scaling:

1)

Kafka > Config > consumer.request.timeout.ms to 500000

Kafka > Config > session.timeout.ms to 60000

Kafka > Config > buffer.memory to 66000000

Kafka > Config > replica.fetch.max.bytes to 56700160

Kafka > Config > message.max.bytes to 56700160

2) You can increase the heartbeat.interval.ms to 20 secs in line with session.timeout.ms (1/3rd of session time). This has to be set in CM > Kafka > Config > Kafka MirrorMaker Advanced Configuration Snippet (Safety Valve) for mirror_maker_consumers.properties > heartbeat.interval.ms = 20000 > Save changes > Restart MM.

3) Add

partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor

to Kafka safety value for mirror.maker.consumer.properties

If you still face the same issue, then provide us the following details for further investigation:

1) Are you getting any errors in the broker logs and MM logs? OR we need broker and MM logs

2) Run in each broker on the destination cluster a couple of times every 5 hours below to discard networking issues:

3) How many brokers are there in the Kafka cluster?

4) Was it working fine before and have you made any recent changes?

Thank you.

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.