How can I configure Kafka (HDP 2.3) on AWS to receive external calls?

[#1]

Hello,

I'm using Kafka 0.8.2.2 (HDP v2.3) and I have my cluster running on Amazon (EC2).

I want to produce messages from external (using Java APIs) into Kafka.

If I execute the code inside the cluster it works, but not from external.

How can I configure it to make it works?

Thanks a lot!

[#2 · Accepted solution]

did you try configuring external hostnames of ec2 boxes in advertised.listeners config.

[#3]

I customised the kafka broker, adding these parameters:

advertised.host.name= public IP on AWS

advertised.port= listener port (for HDP should be 6667)

host.name= public IP on AWS

Now it works!!

Thanks!

[#4]

Thanks Riccardo, it worked for me.

[#5]

Can make it work with below config

With default port

advertised.host.name= 34.207.113.18(public ip of kafka node)

advertised.port= 6667(Listener port)

host.name= 34.207.113.18(public ip of kafka node)

Without default port

listener.security.protocol.map=INGEST:PLAINTEXT,REPLICATION:PLAINTEXT,INTERNAL_PLAINTEXT:PLAINTEXT listeners=REPLICATION://10.0.0.207:9093,INTERNAL_PLAINTEXT://10.0.0.207:9092 advertised.listeners=REPLICATION://34.207.113.18:9093,INTERNAL_PLAINTEXT://34.207.113.18:9092 inter.broker.listener.name=REPLICATION

advertised.port= 9094