Thank you for the nice cheat sheet. I configured according to the cheat sheet above on secure HDP 2.4.2 + Ambari 2.2 cluster. I could send the messages messages using console producer. <Broker_home>/bin/kafka-console-producer.sh --broker-list <KAFKA_BROKER>:6667 --topic test --security-protocol SASL_PLAINTEXT When I am trying to consume the messages (on the same machine) I get error. Starting like this : /usr/hdp/current/kafka-broker/bin/kafka-console-consumer.sh --zookeeper ZK1:2181,ZK-2:2181,ZK-3:2181 --topic test --from-beginning --security-protocol SASL_PLAINTEXT Error stack is : [2016-09-08 13:59:26,167] WARN [console-consumer-39119_HOST_NAME-1473343165849-9f1b8f0d-leader-finder-thread], Failed to find leader for Set([test,0], [test,1]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread) kafka.common.BrokerEndPointNotAvailableException: End point PLAINTEXT not found for broker 0 at kafka.cluster.Broker.getBrokerEndPoint(Broker.scala:141) at kafka.utils.ZkUtils$$anonfun$getAllBrokerEndPointsForChannel$1.apply(ZkUtils.scala:180) at kafka.utils.ZkUtils$$anonfun$getAllBrokerEndPointsForChannel$1.apply(ZkUtils.scala:180) What do you think has gone wrong? Regards, SS PS : Do we have another WIKI page for Best practices around Kafka? @Sriharsha Chintalapani, @Andrew Grande, @Vadim Vaks , @Predrag Minovic, @ ... View more

Thanks for the document and high level comparison. Do you have cost comparision also if we run HDI on monthly basis for 30 days against same of HDP on Azure with similar configuration? ... View more