Getting TOPIC_AUTHORIZATION_FAILED error on kafka producer command

[#1]

Installed kafka broker in a node using ambari blueprint with hdp 2.6.5 with ranger-kafka plugin enabled. Created topics using kafka-topics.sh command. Getting TOPIC_AUTHORIZATION_FAILED warning and unable to produce messages. By going through this document (https://cwiki.apache.org/confluence/display/RANGER/Kafka+Plugin#KafkaPlugin-Authorizingtopiccreation ), got to know that ranger-kafka plugin authorization works only for autocreated topics. Tried the same producer command by auto-created topics but still getting the same error. Could anyone please help me to resolve this.

[#2]

@Shilpa Gokul

There is information that you should have provided to help members help you resolve your problem.

Is you cluster kerberized?

What is the command you are executing? Can you share your Ranger/ kafka policy configuration?

[#3]

@Geoffrey Shelton Okot

Its not a kerberized cluster. Also could you please tell me if there is any way to enable and use ranger with kafka in a non-kerberized cluster.

Screenshots of ranger policy:

[#4]

@Shilpa Gokul

Is the plugin enabled for kafka?

[#5 · Accepted solution]

@Shilpa Gokul

Please have a look at this HCC document by Neeraj Sabharwal how to setup kafka/ranger without Kerberos

This should still be valid with a few tweaks

[#6]

@Geoffrey Shelton Okot

Yes i have enabled ranger-kafka plugin. Have tried with this document also (https://community.hortonworks.com/articles/17059/apache-ranger-and-kafka-1.html ). Created 3 policies for broker, producer and consumer as mentioned in that document, but still getting below error message on running the following producer command(./bin/kafka-console-producer.sh --broker-list hadoop-kafka0:6667 --topic test_2).

hadoop-kafka0 is the hostname of my broker. Tried by giving ip instead of hostname. That also thrown the same error.

"[2019-04-23 10:50:26,907] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 15 : {test_2=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)"

[#7]

@Geoffrey Shelton Okot

Thanks a lot. It worked.

When i created topics using kafka-topics.sh, it throwed LEADER_NOT_AVAILABLE on giving producer command.

So i tried with auto-created topics and it worked without any error.

[#8]

@Shilpa Gokul

If you found this answer addressed your question, please take a moment to log in and click the "Accept" link on the answer.

That would be a great help to Community users to find the solution quickly for these kinds of errors.

[#9]

Very intresting question!