Support Questions
Find answers, ask questions, and share your expertise

kafka broker with multiple NICS



We are running a hadoop 2.6.1 cluster and we are configured two networks:

a) bond WAN>

b) bond DATA >

The hadoop nodes has one network interface with the same last octect in every network.

The hostname is on the host files of the nodes.

It is impossible to make the kafka work. I got always this message when i try to connect to the kafka producer broker from another machine. Firewall is ok.

[2018-03-15 15:09:50,924] ERROR Error when sending message to topic kafka_mirror_test with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
  • it is suited Kafka to receive all the client request from a interface and do the inter broker network from another interface?
  • It is a problem with zookeeper?

HDP 2.6.1 running all the services (hdfs, hive, kafka, pig, zookeeper, etc) in every node.