Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka weird configuration generated by Ambari when SASL_SSL enabled.

Kafka weird configuration generated by Ambari when SASL_SSL enabled.

New Contributor

When generating configuration for Kafka in Apache Ambari 2.7.3.0 with below variables:

security.inter.broker.protocol - SASL_SSL

listeners - SASL_SSL://hadoop1.example.com:6669,SSL://hadoop1.example.com:7000

advertised_listeners - hadoop1.example.com:6669, hadoop1.example.com:7000

the actual server.properties file contain wrong values:

[root@hadoop1 ~]# egrep "listeners=|security.inter" /usr/hdp/current/kafka-broker/conf/server.properties advertised.listeners=SASL_SSL://hadoop1.example.com:6669,SASL_SSL://hadoop1.example.com:7000 advertised_listeners=hadoop1.example.com:6669,hadoop1.example.com:7000 listeners=SASL_SSL://hadoop1.example.com:6669,SASL_SSL://hadoop1.example.com:7000 security.inter.broker.protocol=SASL_SSL

which causes Kafka broker to crash due to invalid configuration. Exactly the same configuration only with

security.inter.broker.protocol - SSL creates proper entities in config file:

[root@hadoop1 ~]# egrep "listeners=|security.inter" /usr/hdp/current/kafka-broker/conf/server.properties advertised_listeners=hadoop1.example.com:6669,hadoop1.example.com:7000 listeners=SASL_SSL://hadoop1.example.com:6669,SSL://hadoop1.example.com:7000 security.inter.broker.protocol=SSL
2 REPLIES 2

Re: Kafka weird configuration generated by Ambari when SASL_SSL enabled.

Mentor

@Leszek Liberda

Is your cluster kerberized? if not that's the reason your config is failing , if yes then check that you have enable the following:

Enable the SASL mechanisms in server.properties:

# List of enabled mechanisms, can be more than one

sasl.enabled.mechanisms=GSSAPI,PLAIN

Specify the SASL security protocol and mechanism for inter-broker communication in server.properties if required:

# Configure SASL_SSL if SSL encryption is enabled, otherwise configure SASL_PLAINTEXT

security.inter.broker.protocol=SASL_SSL

# Configure the appropriate inter-broker protocol

sasl.mechanism.inter.broker.protocol=GSSAPI

Updated the JAAS config file

 KafkaServer {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   storeKey=true
   keyTab="/etc/security/keytabs/kafka_server.keytab"
   principal="kafka/kafka1.hostname.com@EXAMPLE.COM";

Please revert

Re: Kafka weird configuration generated by Ambari when SASL_SSL enabled.

New Contributor

Hi, The cluster is fully kerberized and encrypted with SSL. We haven't observed this misbehave when only one ( SASL_SSL ) listener is configured

security.inter.broker.protocol=SASL_SSL
listeners=SASL_SSL://hadoop1.example.com:6669

or with one plaintext listener and one ssl listener ( which also works ):

security.inter.broker.protocol=SASL_PLAINTEXT
listeners=SASL_PLAINTEXT://hadoop1.example.com:6669,SSL://hadoop1.example.com:7000

As soon as we try to use SASL_SSL and SSL listeners the SSL entry from ambari becomes SASL_SSL entry in server.properties

Don't have an account?
Coming from Hortonworks? Activate your account here