Support Questions
Find answers, ask questions, and share your expertise

Kafka console producer from one cluster to another (centos with confluent to HDP 2.4) - error when sending a message

Explorer

Hello Guys,

I have two machines. First one (1) is virtual machine on my personal laptop, with CentOS 7 and Confluent Open Source installed on it. Second one (2) is a virtual server within internal network with HDP 2.4 installed.

I am trying to produce some messages from CentOS (1) to HDP (2) machine using command:

[root@localhost bin]# /bin/kafka-console-producer --broker-list http://hdphostip:6667 --topic test
>123

but I am unable to do so, due to the following error.

[2017-09-15 10:19:23,983] ERROR Error when sending message to topic test with key: null, value: 3 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

I browsed google with several topics and tried different broker addresses like PLAINTEXT://hdphostip:6667 etc, but nothing seems to be working.

Oh HDP i have the following listener set:

PLAINTEXT://localhost:6667

i have tried 0.0.0.0 as well, same result.

What's curious is that i was able to create topic using the following zookeeper:

--zookeeper hdphostip:2181

so the ip is reachable and is working fine.

Any ideas how I could wrote messages to mentioned broker?

Thanks in advance.