How to configure external accounts for Streams Replication Manager using 2 kerberized clusters

[#1]

This article provides the steps and fields required to configure Streams Replication Manager using external accounts between two kerberized clusters.

This article assumes that Kerberos is configured properly between the clusters and we can produce and consume data correctly.

Environment details:

• Cluster A: co-located cluster (SRM and Kafka running in this cluster) CDP 7.1.7
• Cluster B: External cluster which is another CDP 7.1.7
• Both clusters are using  SASL_PLAINTEXT security.protocol for their clients
• Clusters are configured with cross-domain realm (MIT kerberos), more details on cross-domain realm configuration here

Steps to configure external accounts (feature available from CDP 7.1.7)

1. Go to cluster A > Cloudera Manager > Administration > External accounts > Kafka Credentials tab
2. Configure the following fields:
   
   Name
   Bootstrap servers
   Security protocol
   JAAS Secret [1-3]
   JAAS Template
   Kerberos Service Name
   SASL Mechanism

   Example:

   Name	c289
   Bootstrap servers	c289-node2.clusterB.com:9092,c289-node3.clusterB.com:9092
   Security protocol	SASL_PLAINTEXT
   JAAS Secret 1	kafka/c189-node4.clusterA.com@CLUSTERA.COM
   JAAS Secret 2	/opt/cloudera/kafka.keytab
   JAAS Template	com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab="##JAAS_SECRET_1##" principal="##JAAS_SECRET_2##";
   Kerberos Service Name	kafka
   SASL Mechanism	GSSAPI

   
   
   • In the JAAS secret fields, we can also use "username", "password" and replace the Krb5LoginModule with PlainLoginModule 
   • Also, make sure that if we are using more than one srm driver, copy the keytab to each srm-driver host with the correct permissions (SRM PID owner), for example:
     -rw------- 1 streamsrepmgr streamsrepmgr 216 Jan 28 14:32 kafka.keytab
   • We can also configure an external account for the co-located cluster, but this is not required
     
   • * values means not required
     
     
3. Finally, go to cluster A > Cloudera Manager > Streams Replication Manager > Configuration and add the external account name c289 in the External Kafka Accounts field:
   
   
4. Configure the replication details under cluster A > Cloudera Manager > Streams Replication Manager > Configuration > Streams Replication Manager's Replication Configs
5. Start the SRM cluster and validate that the properties are correct in the srm-driver.log files.

Additional details about SRM configuration Configuring Streams Replication Manager.