question Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command in Support Questions

Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

gshilpa3311 — Fri, 19 Apr 2019 11:44:36 GMT

Installed kafka broker in a node using ambari blueprint with hdp 2.6.5 with ranger-kafka plugin enabled. Created topics using kafka-topics.sh command. Getting TOPIC_AUTHORIZATION_FAILED warning and unable to produce messages. By going through this document (https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin#KafkaPlugin-Authorizingtopiccreation ), got to know that ranger-kafka plugin authorization works only for autocreated topics. Tried the same producer command by auto-created topics but still getting the same error. Could anyone please help me to resolve this.

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

Shelton — Sun, 21 Apr 2019 16:44:48 GMT

@Shilpa Gokul

There is information that you should have provided to help members help you resolve your problem.

Is you cluster kerberized?

What is the command you are executing? Can you share your Ranger/ kafka policy configuration?

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

gshilpa3311 — Sat, 17 Aug 2019 22:51:28 GMT

@Geoffrey Shelton Okot

Its not a kerberized cluster. Also could you please tell me if there is any way to enable and use ranger with kafka in a non-kerberized cluster.

Screenshots of ranger policy:

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

Shelton — Mon, 22 Apr 2019 22:55:01 GMT

@Shilpa Gokul

Is the plugin enabled for kafka?

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

Shelton — Tue, 23 Apr 2019 11:12:26 GMT

@Shilpa Gokul

Please have a look at this HCC document by Neeraj Sabharwal how to setup kafka/ranger without Kerberos

This should still be valid with a few tweaks

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

gshilpa3311 — Tue, 23 Apr 2019 17:58:23 GMT

@Geoffrey Shelton Okot

Yes i have enabled ranger-kafka plugin. Have tried with this document also (https://community.hortonworks.com/articles/17059/apache-ranger-and-kafka-1.html ). Created 3 policies for broker, producer and consumer as mentioned in that document, but still getting below error message on running the following producer command(./bin/kafka-console-producer.sh --broker-list hadoop-kafka0:6667 --topic test_2).

hadoop-kafka0 is the hostname of my broker. Tried by giving ip instead of hostname. That also thrown the same error.

"[2019-04-23 10:50:26,907] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 15 : {test_2=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)"

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

gshilpa3311 — Tue, 23 Apr 2019 20:12:22 GMT

@Geoffrey Shelton Okot

Thanks a lot. It worked.

When i created topics using kafka-topics.sh, it throwed LEADER_NOT_AVAILABLE on giving producer command.

So i tried with auto-created topics and it worked without any error.

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

Shelton — Tue, 23 Apr 2019 21:58:20 GMT

@Shilpa Gokul

If you found this answer addressed your question, please take a moment to log in and click the "Accept" link on the answer.

That would be a great help to Community users to find the solution quickly for these kinds of errors.

Re: Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

jamessweney — Wed, 24 Apr 2019 03:31:21 GMT

Very intresting question!