question Re: How can I configure Kafka (HDP 2.3) on AWS to receive external calls? in Archives of Support Questions (Read Only)

How can I configure Kafka (HDP 2.3) on AWS to receive external calls?

riccardo_rappol — Mon, 04 Apr 2016 15:42:32 GMT

Hello,

I'm using Kafka 0.8.2.2 (HDP v2.3) and I have my cluster running on Amazon (EC2).

I want to produce messages from external (using Java APIs) into Kafka.

If I execute the code inside the cluster it works, but not from external.

How can I configure it to make it works?

Thanks a lot!

Re: How can I configure Kafka (HDP 2.3) on AWS to receive external calls?

schintalapani — Tue, 05 Apr 2016 01:17:39 GMT

did you try configuring external hostnames of ec2 boxes in advertised.listeners config.

Re: How can I configure Kafka (HDP 2.3) on AWS to receive external calls?

riccardo_rappol — Wed, 18 May 2016 16:05:14 GMT

I customised the kafka broker, adding these parameters:

advertised.host.name= public IP on AWS

advertised.port= listener port (for HDP should be 6667)

host.name= public IP on AWS

Now it works!!

Thanks!

Re: How can I configure Kafka (HDP 2.3) on AWS to receive external calls?

sivasankar_boom — Fri, 29 Jun 2018 17:29:34 GMT

Thanks Riccardo, it worked for me.

Re: How can I configure Kafka (HDP 2.3) on AWS to receive external calls?

sivasankar_boom — Fri, 17 Aug 2018 11:19:36 GMT

Can make it work with below config

With default port

advertised.host.name= 34.207.113.18(public ip of kafka node)

advertised.port= 6667(Listener port)

host.name= 34.207.113.18(public ip of kafka node)

Without default port

listener.security.protocol.map=INGEST:PLAINTEXT,REPLICATION:PLAINTEXT,INTERNAL_PLAINTEXT:PLAINTEXT listeners=REPLICATION://10.0.0.207:9093,INTERNAL_PLAINTEXT://10.0.0.207:9092 advertised.listeners=REPLICATION://34.207.113.18:9093,INTERNAL_PLAINTEXT://34.207.113.18:9092 inter.broker.listener.name=REPLICATION

advertised.port= 9094