Kafka console-consumer/-producer not working after adding SASL_PLAINTEXT

[#1]

Hi,

after enabling SASL_PLAINTEXT listener on kafka it is no longer possible to use console-consumer/-producer. Whereas using a simple Java snippet to create a producer and adding some messages, it works fine, by using the exact same user/password as used for the console-clients:

public class SimpleProducer {
    public static void main(String[] args) throws Exception{
        if(args.length == 0){
            System.out.println("Enter topic name");
            return;
        }
        String topicName = args[0].toString();
        Properties props = new Properties();
        props.put("bootstrap.servers", "<brokernode>:6666");
        props.put("acks", "1");
        props.put("retries", 0);
        props.put("batch.size", 16384);
        props.put("linger.ms", 1);
        props.put("buffer.memory", 33554432);
        props.put("key.serializer",
            "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer",
            "org.apache.kafka.common.serialization.StringSerializer");
        ////// AUTHENTICATION
        props.put("security.protocol","SASL_PLAINTEXT");
        props.put("sasl.mechanism","PLAIN");
        props.put("sasl.jaas.config",
            "org.apache.kafka.common.security.plain.PlainLoginModule required\n" +
            "username=\"kafka\"\n" +
            "password=\"kafkaSecure\";");

        ////// END AUTHENTICATION
        Producer<String, String> producer = new KafkaProducer<String, String>(props);
        System.out.println("producer created");
        for(int i = 0; i < 10; i++) {
            System.out.println("message"+i);
            producer.send(new ProducerRecord<String, String>(topicName,
                Integer.toString(i), Integer.toString(i)));
        }
        System.out.println("Messages sent successfully");
        producer.close();
    }
}

After starting the e.g. producer and trying to add a message via the console, the following message is shown (endless):

[2017-11-14 16:48:23,039] WARN Bootstrap broker <brokernode>:6666 disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 16:48:23,091] WARN Bootstrap broker <brokernode>:6666 disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 16:48:23,143] WARN Bootstrap broker <brokernode>:6666 disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 16:48:23,195] WARN Bootstrap broker <brokernode>:6666 disconnected (org.apache.kafka.clients.NetworkClient)

Kafka config looks like:

listeners=PLAINTEXT://<brokernode>:6667,SASL_PLAINTEXT://<brokernode>:6666
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
security.inter.broker.protocol=SASL_PLAINTEXT

The console-producer gets started via:

export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/conf/user_kafka_jaas.conf" ; /usr/hdf/current/kafka-broker/bin/kafka-console-producer.sh --broker-list <brokernode>:6666 --topic gk-test --producer.config /etc/kafka/conf/producer.properties

where the property files look like:

/etc/kafka/conf/user_kafka_jaas.conf

KafkaClient {
  org.apache.kafka.common.security.plain.PlainLoginModule required
  username="kafka"
  password="kafkaSecure";
};

/etc/kafka/conf/producer.properties

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN

Any hint on what is going wrong with console-producer and console-consumer to not being able to produce/consume from topic ? ...but the Java snippet works...

Thanks

[#2]

I think your problem is similar to this one

https://community.hortonworks.com/questions/120978/hdp-kafka-service-setup-with-saslplain-not-workin...

I did what @Jeff Groves said, it worked. You might have a try.

[#3]

@Gerd Koenig

Is your broker using the jaas (username/password) you created?

The SASL/PLAIN configuration is well documented here, Do ensure the username and password are in the jaas which broker loading in classpath.