Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

8 replies
34065 views
Translate All

Installed kafka broker in a node using ambari blueprint with hdp 2.6.5 with ranger-kafka plugin enabled. Created topics using kafka-topics.sh command. Getting TOPIC_AUTHORIZATION_FAILED warning and unable to produce messages. By going through this document (https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin#KafkaPlugin-Authorizingtopiccreation ), got to know that ranger-kafka plugin authorization works only for autocreated topics. Tried the same producer command by auto-created topics but still getting the same error. Could anyone please help me to resolve this.

SheltonAnswer

Gold ⭐️⭐️⭐️

@Shilpa Gokul

Please have a look at this HCC document by Neeraj Sabharwal how to setup kafka/ranger without Kerberos

This should still be valid with a few tweaks

Shelton

Gold ⭐️⭐️⭐️

@Shilpa Gokul

There is information that you should have provided to help members help you resolve your problem.

Is you cluster kerberized?

What is the command you are executing? Can you share your Ranger/ kafka policy configuration?

Translate

gshilpa3311Author

Silver ⭐️⭐️

@Geoffrey Shelton Okot

Its not a kerberized cluster. Also could you please tell me if there is any way to enable and use ranger with kafka in a non-kerberized cluster.

Screenshots of ranger policy:

Translate

Shelton

Gold ⭐️⭐️⭐️

@Shilpa Gokul

Is the plugin enabled for kafka?

Translate

SheltonAnswer

Gold ⭐️⭐️⭐️

@Shilpa Gokul

Please have a look at this HCC document by Neeraj Sabharwal how to setup kafka/ranger without Kerberos

This should still be valid with a few tweaks

Translate

gshilpa3311Author

Silver ⭐️⭐️

@Geoffrey Shelton Okot

Yes i have enabled ranger-kafka plugin. Have tried with this document also (https://community.hortonworks.com/articles/17059/apache-ranger-and-kafka-1.html ). Created 3 policies for broker, producer and consumer as mentioned in that document, but still getting below error message on running the following producer command(./bin/kafka-console-producer.sh --broker-list hadoop-kafka0:6667 --topic test_2).

hadoop-kafka0 is the hostname of my broker. Tried by giving ip instead of hostname. That also thrown the same error.

"[2019-04-23 10:50:26,907] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 15 : {test_2=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)"

Translate