I'm using HDP 2.3.4 with kafka 0.9 The kafka console producer/consumer worked well, but when I tried to create a simple kafka producer using scala in windows, it went into exception with error description in subject. Scala code was some sample code from git, as below: object ProducerExample extends App { import java.util.Properties import org.apache.kafka.clients.producer._ val props = new Properties() props.put("bootstrap.servers", "serverX:6667") props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer") props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer") props.put("security.protocol", "SASL_PLAINTEXT") val producer = new KafkaProducer[String, String](props) val TOPIC="test" for(i<- 1 to 50){ val record = new ProducerRecord(TOPIC, "key", s"hello $i") producer.send(record) } val record = new ProducerRecord(TOPIC, "key", "the end "+new java.util.Date) producer.send(record) producer.close() } I'm pretty sure the bootstrap.servers setting is correct, as the console producer/consumer works with the same value. Any comments/clues? thanks a lot! ... View more

I'm using HDP 2.3.4 with kafka 0.9 I just started to use kafka referring to this document, but having problem with the kafka-console-consumer. below are the commands used to create a topic and publish messages bin/kafka-topics.sh --zookeeper <serverX>:2181 --create --topic test2 --partitions 2 --replication-factor 1 bin/kafka-console-producer.sh --broker-list <brokerX>:6667 --topic test2 --security-protocol SASL_PLAINTEXT both commands worked well. Tried kafka simple consumer, and worked well, message were read and displayed bin/kafksimple-consumer-shell.sh --broker-list <brokerX>:6667 --topic test2 --security-protocol SASL_PLAINTEXT --partition 0 bin/kafksimple-consumer-shell.sh --broker-list <brokerX>:6667 --topic test2 --security-protocol SASL_PLAINTEXT --partition 1 but I'm not able to get kafka-console-consumer work. Tried below command first, but hitting exceptions: bin/kafka-console-consumer.sh --zookeeper <serverX>:2181 --topic test2 --from-beginning --security-protocol SASL_PLAINTEXT WARN [console-consumer-89015_xxx-1475146648772-75b1a67a-leader-finder-thread], Failed to find leader for Set([test2,0], [test2,1]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread) kafka.common.BrokerEndPointNotAvailableException: End point SASL_PLAINTEXT not found for broker 1002 some online post suggested to use new-consumer option for kafka 0.9, so tried the command below: bin/kafka-console-consumer.sh --zookeeper <serverX>:2181 --topic test2 --bootstrap-server <serverY>:9092 --new-consumer --security-protocol SASL_PLAINTEXT consumer started without error, but no messages were read and displayed. I'm not sure what went wrong, any suggestions? Thanks a lot! ... View more