Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Topic creation and deletion are not protected after enabling Kerberos in Kafka

avatar
Contributor

I have enabled Kerberos authentication for Kafka as the documentation suggests and indeed producing to topics and consuming from topics requires authentication. Surprisingly, topic creation and deletion do not require authentication. Could somebody tell me whether this goes wrong ? Really appreciate it.

 

CDH version: 5.15.1

CDK version: 4.1.0

test command:

 

 

kafka-topics --create --zookeeper <zookeeper-host>:2181 --replication-factor 2 --partitions 3 --topic test2
kafka-topics --delete --zookeeper <zookeeper-host>:2181 --topic test2

 

 

 

1 ACCEPTED SOLUTION

avatar
Rising Star

Hi @iamabug 

It's a known limitation in Kafka where the kafka-topics tool communicates directly with Zookeeper. When you create a topic, all the tool does is connect to Zookeeper, creates a znode representing this topic and then sets some data as a JSON string (the metadata for the topic).

There has been work to develop Java admin clients which made some progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+...

However, all that's left is to have command line tools that leverage those Java APIs and that's a work in progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrativ...


View solution in original post

8 REPLIES 8

avatar
Rising Star

Hi @iamabug 

It's a known limitation in Kafka where the kafka-topics tool communicates directly with Zookeeper. When you create a topic, all the tool does is connect to Zookeeper, creates a znode representing this topic and then sets some data as a JSON string (the metadata for the topic).

There has been work to develop Java admin clients which made some progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+...

However, all that's left is to have command line tools that leverage those Java APIs and that's a work in progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrativ...


avatar
Master Mentor

@iamabug 

There is a lot more than just kerberizing the cluster and you are good to go. Have you enabled SSL also? Can you share a tokenized version of the below files? Basically, the ACL in zk is the key to who can do what and usually the Kafka admin is the only one allowed!

 

  • server.properties [listeners, advertised.listeners,authorizer.class.name,sasl.enabled.mechanism and super.users]
  • Kafka_server_jaas.conf
  • Kafka_client_jaas.conf
  • kafka_client_kerberos.properties

Hope that helps

avatar
Contributor

Thanks for your answer. I now believe that ACL in Zookeeper may be the solution here.

avatar
Master Mentor

@iamabug 

Are you now comfortable proceeding? If you need some help don't hesitate to ask.

avatar
Contributor

It's really nice of you. I would definitely ask for your help when something tricky comes up. Thank you very much.

avatar

@Shelton I have the same problem but with cloudera, do you know what procedure I should follow to configure the zookeeper ACL but with kafka and sentry? Thank you

avatar

@WilsonLozano,

 

As this thread is older and was marked 'Solved back in August of 2019 you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment, version of CDH, etc. that could aid others in providing a more accurate answer to your question. 

 

 

Bill Brooks, Community Moderator
Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Contributor

Thanks.