Support Video: How to run Kafka Mirror Maker using Kerberos clusters?

This video provides the steps to configure Kafka Mirror Maker in Kerberized clusters.

Open YouTube video here

Environment:

Cluster A (Source-Kerberized):

c189-node2.squadron-labs.com Broker1
c189-node3.squadron-labs.com Broker2
c189-node4.squadron-labs.com Broker3

Cluster B (Destination-Kerberized):

c289-node2.squadron-labs.com Broker1
c289-node3.squadron-labs.com Broker2
c289-node4.squadron-labs.com Broker3

Destination Files 

consumer.properties
bootstrap.servers=<brokerSourceHost>:<brokerPort>,<brokerSourceHost>:<brokerPort>
group.id=<consumerGroupName>
security.protocol=PLAINTEXTSASL

producer.properties
bootstrap.servers=<brokerDestinationHost>:<brokerPort>,<brokerDestinationHost>:<brokerPort>
key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer
security.protocol=PLAINTEXTSASL

kafka_mirrormaker_jaas.conf
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="<pathTokeytab>"
storeKey=true
useTicketCache=false
serviceName="kafka"
principal="<principal>@<REALM>";
};

Destination commands

1. export KAFKA_OPTS="-Djava.security.auth.login.config=<path_to_kafka_mirrormaker_jaas.conf>"

2. ./kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config <path_To_Consumer.properties>
--producer.config <path_To_producer.properties> --whitelist "<MirrorMakertopicName>"

Number of consumption streams

Use the --num.streams option to specify the number of mirror consumer threads to create. 

Source commands

1. ./kafka-console-producer.sh --broker-list <brokerSourceHost>:<brokerPort>
--topic <MirrorMakertopicName> --security-protocol PLAINTEXTSASL

2. ./kafka-console-consumer.sh --bootstrap-server <brokerDestinationHost>:<brokerPort>
--topic <MirrorMakertopicName> --security-protocol PLAINTEXTSASL