Topic creation and deletion are not protected after enabling Kerberos in Kafka

[#1]

I have enabled Kerberos authentication for Kafka as the documentation suggests and indeed producing to topics and consuming from topics requires authentication. Surprisingly, topic creation and deletion do not require authentication. Could somebody tell me whether this goes wrong ? Really appreciate it.

CDH version: 5.15.1

CDK version: 4.1.0

test command:

kafka-topics --create --zookeeper <zookeeper-host>:2181 --replication-factor 2 --partitions 3 --topic test2
kafka-topics --delete --zookeeper <zookeeper-host>:2181 --topic test2

[#2 · Accepted solution]

Hi @iamabug 

It's a known limitation in Kafka where the kafka-topics tool communicates directly with Zookeeper. When you create a topic, all the tool does is connect to Zookeeper, creates a znode representing this topic and then sets some data as a JSON string (the metadata for the topic).

There has been work to develop Java admin clients which made some progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+...

However, all that's left is to have command line tools that leverage those Java APIs and that's a work in progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrativ...

[#3]

@iamabug 

There is a lot more than just kerberizing the cluster and you are good to go. Have you enabled SSL also? Can you share a tokenized version of the below files? Basically, the ACL in zk is the key to who can do what and usually the Kafka admin is the only one allowed!

• server.properties [listeners, advertised.listeners,authorizer.class.name,sasl.enabled.mechanism and super.users]
• Kafka_server_jaas.conf
• Kafka_client_jaas.conf
• kafka_client_kerberos.properties

Hope that helps

[#4]

Thanks for your answer. I now believe that ACL in Zookeeper may be the solution here.

[#5]

@iamabug 

Are you now comfortable proceeding? If you need some help don't hesitate to ask.

[#6]

It's really nice of you. I would definitely ask for your help when something tricky comes up. Thank you very much.

[#7]

@Shelton I have the same problem but with cloudera, do you know what procedure I should follow to configure the zookeeper ACL but with kafka and sentry? Thank you

[#8]

@WilsonLozano,

As this thread is older and was marked 'Solved back in August of 2019 you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment, version of CDH, etc. that could aid others in providing a more accurate answer to your question. 

[#9]

Thanks.