question Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka in Support Questions

Topic creation and deletion are not protected after enabling Kerberos in Kafka

iamabug — Fri, 23 Aug 2019 16:10:24 GMT

I have enabled Kerberos authentication for Kafka as the documentation suggests and indeed producing to topics and consuming from topics requires authentication. Surprisingly, topic creation and deletion do not require authentication. Could somebody tell me whether this goes wrong ? Really appreciate it.

CDH version: 5.15.1

CDK version: 4.1.0

test command:

kafka-topics --create --zookeeper <zookeeper-host>:2181 --replication-factor 2 --partitions 3 --topic test2 kafka-topics --delete --zookeeper <zookeeper-host>:2181 --topic test2

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

w@leed — Fri, 23 Aug 2019 13:45:22 GMT

Hi @iamabug

It's a known limitation in Kafka where the kafka-topics tool communicates directly with Zookeeper. When you create a topic, all the tool does is connect to Zookeeper, creates a znode representing this topic and then sets some data as a JSON string (the metadata for the topic).

There has been work to develop Java admin clients which made some progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-117%3A+Add+a+public+AdminClient+API+for+Kafka+admin+operations#KIP-117:AddapublicAdminClientAPIforKafkaadminoperations-FutureWork

However, all that's left is to have command line tools that leverage those Java APIs and that's a work in progress:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-4+-+Command+line+and+centralized+administrative+operations

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

Shelton — Sun, 25 Aug 2019 18:59:51 GMT

@iamabug

There is a lot more than just kerberizing the cluster and you are good to go. Have you enabled SSL also? Can you share a tokenized version of the below files? Basically, the ACL in zk is the key to who can do what and usually the Kafka admin is the only one allowed!

server.properties [listeners, advertised.listeners,authorizer.class.name,sasl.enabled.mechanism and super.users]
Kafka_server_jaas.conf
Kafka_client_jaas.conf
kafka_client_kerberos.properties

Hope that helps

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

iamabug — Mon, 26 Aug 2019 07:46:01 GMT

Thanks.

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

iamabug — Mon, 26 Aug 2019 07:50:06 GMT

Thanks for your answer. I now believe that ACL in Zookeeper may be the solution here.

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

Shelton — Mon, 26 Aug 2019 12:03:19 GMT

@iamabug

Are you now comfortable proceeding? If you need some help don't hesitate to ask.

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

iamabug — Mon, 26 Aug 2019 12:07:17 GMT

It's really nice of you. I would definitely ask for your help when something tricky comes up. Thank you very much.

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

WilsonLozano — Wed, 19 Feb 2020 20:11:58 GMT

@Shelton I have the same problem but with cloudera, do you know what procedure I should follow to configure the zookeeper ACL but with kafka and sentry? Thank you

Re: Topic creation and deletion are not protected after enabling Kerberos in Kafka

ask_bill_brooks — Thu, 20 Feb 2020 00:44:52 GMT

@WilsonLozano,

As this thread is older and was marked 'Solved back in August of 2019 you would have a better chance of receiving a resolution by starting a new thread. This will also provide the opportunity to provide details specific to your environment, version of CDH, etc. that could aid others in providing a more accurate answer to your question.