Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

What is Kafka Gateway and Kafka MirrorMaker when add Role Instances to Kafka?

avatar
Contributor

What is Kafka Gateway and Kafka MirrorMaker when add Role Instances to Kafka?

I created a Kafka service on Cloudera Manager. Now I want to add a new Kafka broker instance inside the Kafka service. I followed the guide on the internet. Choose Instances -> Add Role Instances

quangbilly79_0-1672296142699.png

A new window comes up. But I notice that I can only add Kafka Brokers Instances (this guide said that I can add Kafka Connect Instance too).

Also, there are two other instances called Gateway and Mirrormaker, which I don't know what are they? I search google but only find some info about Kafka MirrorMaker but no luck on finding anything about Kafka Gateway

quangbilly79_1-1672296496410.png

 

 

1 ACCEPTED SOLUTION

avatar
Super Collaborator

Hello @quangbilly79 

 

Thanks for using Cloudera Community. Based on your Post, you may consider "Kafka Gateway" as the Client for Kafka, which are setup on the Hosts wherein the same is added as per Cloudera Manager "Assign Roles". A Client/Gateway is familiar with the Service (Kafka in this Case) & all Client/Service Configs are available for the Client/Gateway without any manual intervention. Any changes made to the Service or Client Configs is pushed to the Service/Client Configuration by Cloudera Manager. 

 

Imagine a Scenario wherein you wish to run "hdfs dfs -ls" on a HDFS FileSystem. Simply running the Command won't work unless the Host wherein the Command "hdfs dfs -ls" is being run knows the Setup (HDFS FileSystem, NameNode, Port, Protocol). Review [1] for an Example. Adding an HDFS Gateway ensures User doesn't need to manually configure a Client/Gateway with Cloudera Manager doing the needful. 

 

Similarly, Kafka Gateway operates. Else, Customer need to manually configure the Client/Gateway Setup. Hope the above answer your query concerning the Gateway Role. 

 

Regards, Smarak

 

[1] https://www.ibm.com/docs/en/spectrum-scale-bda?topic=hdfs-clients-configuration

View solution in original post

1 REPLY 1

avatar
Super Collaborator

Hello @quangbilly79 

 

Thanks for using Cloudera Community. Based on your Post, you may consider "Kafka Gateway" as the Client for Kafka, which are setup on the Hosts wherein the same is added as per Cloudera Manager "Assign Roles". A Client/Gateway is familiar with the Service (Kafka in this Case) & all Client/Service Configs are available for the Client/Gateway without any manual intervention. Any changes made to the Service or Client Configs is pushed to the Service/Client Configuration by Cloudera Manager. 

 

Imagine a Scenario wherein you wish to run "hdfs dfs -ls" on a HDFS FileSystem. Simply running the Command won't work unless the Host wherein the Command "hdfs dfs -ls" is being run knows the Setup (HDFS FileSystem, NameNode, Port, Protocol). Review [1] for an Example. Adding an HDFS Gateway ensures User doesn't need to manually configure a Client/Gateway with Cloudera Manager doing the needful. 

 

Similarly, Kafka Gateway operates. Else, Customer need to manually configure the Client/Gateway Setup. Hope the above answer your query concerning the Gateway Role. 

 

Regards, Smarak

 

[1] https://www.ibm.com/docs/en/spectrum-scale-bda?topic=hdfs-clients-configuration