Why Ambari is setting the security protocol of the kafka to PLAINTEXTSASL instead of SASL_PLAINTEXT?

[#1]

Hi All ,

During Kerboraizing the kafka using the Ambari , it is setting the kafka security protocol to PLAINTEXTSASL instead of SASL_PLAINTEXT, but everywhere in the document is it mentioned that it must be SASL_PLAINTEXT , I have few questions regarding this .

1. Why Ambari setting the security protocol to PLAINTEXTSASL , is it a bug ?

2. Even though we are able to produce and consume the messages from program written in java.

But in the producer we are setting the security protocol to PLAINTEXTSASL, and in the consumer SASL_PLAINTEXT , it is working fine , Question is how come it is working fine when actual protocol is just PLAINTEXTSASL.

Thanks in Advance ,

Param.

[#2 · Accepted solution]

@Param NC Kafka security is developed by Hortonworks. Before it shipped into Apache Kafka we shipped it in HDP. At that time we called the SASL protocol as PLAINTEXTSASL which later changed SASL_PLAINTEXT. These protocols are synonymous so you can use them interchangeably. Older version of AMBARI still calls it as PLAINTEXTSASL which will be changing to SASL_PLAINTEXT in upcoming version.

In your case, producer & consumer are working because PLAINTEXTSASL or SASL_PLAINTEXT means the same thing and can be used interchangeably. For consistency purpose , we recommend you to use SASL_PLAINTEXT everywhere.

[#3]

Thanks for the response I very much agree to you answer .

[#4]

Hello Sriharsha, just a quick question. When I use confluentinc kafka rest proxy in company's HDP 2.5.3 cluster, I run into the error "No security protocol defined for listener PLAINTEXTSASL" and "broker info from zookeeper cannot be parsed". I guess that is because PLAINTEXTSASL cannot be recognized by other frameworks, though producer and consumer can work within HDP. Is there any way solving this problem?

[#5]

@Qingyang Kong

Kafka rest proxy uses old client apis, which are not supported in secure cluster in Apache. However HDP kafka supports security in old consumer apis. To enable this you need to build kafka rest proxy code with HDP kafka_core dependency and pass a KafkaClient jaas config to your kafka rest server JVM.

[#6]

We encountered a similar issue when upgrading our Ambari from 2.4 to 2.5. Our Kafka brokers would not restart.

Here was the error message:

/var/log/kafka/server.log.2017-06-27-19:java.lang.IllegalArgumentException: requirement failed: security.inter.broker.protocol must be a protocol in the configured set of advertised.listeners. The valid options based on currently configured protocols are Set(SASL_PLAINTEXT)

We had specified PLAINTEXTSASL as the SASL protocol in the configuration.

To fix this we changed the following configuration in Custom kafka-broker:

security.inter.broker.protocol=SASL_PLAINTEXT