Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka not connecting from localhost(eclipse) to hdp2.6.5 docker image (mac)

Highlighted

Kafka not connecting from localhost(eclipse) to hdp2.6.5 docker image (mac)

Contributor

Hi All,

I am not able to connect Kafka consumer and producer running on a hdp2.6.5 container on mac. I am using eclipse to connect to sandbox-hdp.hortonworks.com:6667. I can see the port forwarding but still not working.

docker ps
$   4f94aca8bc81hortonworks/sandbox-proxy:1.0 "nginx -g 'daemon of…" 2 hours ago Up 2 hours  ........  0.0.0.0:6627->6627/tcp, 0.0.0.0:6667->6667/tcp, 0.0.0.0:7777->7777/tcp, 0.0.0.0:7788->7788/tcp, 0.0.0.0:8000->8000/tcp, 0.0.0.0:8005->8005/tcp, 0.0.0.0:8020->8020/tcp, 0.0.0.0:8032->8032/tcp, 0.0.0.0:8040->8040/tcp, 0.0.0.0:8042->8042/tcp, .......      sandbox-proxy  b76d24c6c5f8  
$ hortonworks/sandbox-hdp:2.6.5  "/usr/sbin/init"  2 hours ago Up 2 hours22/tcp, 4200/tcp, 8080/tcp 
class kafkaProducer {
    val kafkaBrokers = "sandbox-hdp.hortonworks.com:6667"
    val topicName = "test"
    val props = new Properties()
    props.put("bootstrap.servers", kafkaBrokers)
    props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer")
    props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer")
    val producer = new KafkaProducer[String, String](props)
    def sendEvent(message: String)={
      val key = java.util.UUID.randomUUID().toString()
      producer.send(new ProducerRecord[String, String](topicName, key, message))
    }  
}
1 REPLY 1

Re: Kafka not connecting from localhost(eclipse) to hdp2.6.5 docker image (mac)

Super Collaborator

Can you please verify / show the Kafka Server properties?

When you run Kafka within a container, you need to make sure that clients are getting the external addrress from Zookeeper. A simple port forward is not enough.

https://rmoff.net/2018/08/02/kafka-listeners-explained/