In order to communicate in kerberos enabled cluster one needs to authenticate itself. So when broker will try to communicate with other broker in the cluster it will need to authenticate first. Same for the clients communicating to the brokers.
In Kafka, JAAS files is used for authentication. Lets first understand what's there in JAAS file. In kafka there are two JAAS files :
Let's discuss about kafka_jaas.conf. This file will be used for authentication when a broker in a cluster tries to communicate with other brokers in cluster. Take a look at the content and understand its purpose :
KafkaClient section : As the name suggest it will be used when client wants to communicate to broker.
Value of `serviceName` should be the principal as which kafka is running. You can configure it to use ticket cache or keytab and principal.
Client : This part of JAAS file is only used by clients which are using old consumer api. In old consumer api, consumers need to connect to zookeeper.
In new consumer api clients communicate to brokers instead of zookeeper. Hence while authentication it will use KafkaClient section in kafka_client_jaas.conf.
Producers will always use KafkaClient section in kafka_client_jaas.conf as it will send request to broker node.
For long running kafka clients it recommended to configure JAAS file to use keytab and principal.