Support Questions
Find answers, ask questions, and share your expertise

kafka send message error

Solved Go to solution

kafka send message error

Contributor

After enable ranger kafka plugin, execute command "/usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list bigdata001:6667 --topic test5", but when I input content to send message, the result returns as follows:

[2017-03-30 17:06:45,507] WARN Error while fetching metadata with correlation id 0 : {test5=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2017-03-30 17:06:45,507] ERROR Error when sending message to topic test5 with key: null, value: 7 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [test5]
[2017-03-30 17:11:45,563] WARN Error while fetching metadata with correlation id 1 : {test5=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
8 REPLIES 8
Highlighted

Re: kafka send message error

Super Mentor

@Zhao Chaofeng

Please login to Ranger UI http://RANGER_HOST:6080/index.html#!/policymanager/resource

Check the policies defined for Kafka (specially for topics) that you are not restricting access.

.

Purely at Kafka level you can check the permissions using the following utility:

# bin/kafka-acls.sh --list --topic <TOPIC_NAME>

.

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_secure-kafka-ambari/content/kafka-acl-exa...

Highlighted

Re: kafka send message error

Contributor

Yes, I defined a policy in ranger as follows:

14217-selection-049.png

Highlighted

Re: kafka send message error

Contributor
# bin/kafka-acls.sh --list --topic test5

After executing the command, there is no acls for topic test5.

Highlighted
Highlighted

Re: kafka send message error

Contributor

OK, thank you for your reply.

Highlighted

Re: kafka send message error

New Contributor

@sunil thupakulaI

i am also getting the same error when i try to publish/consume messages.

I have created a topic with one partition, using kafka user , though i have 3 brokers in my kafka cluster, i have also authorized using ranger policy . When ever i alter the topic partitions to more than 1 then i get below error.


ERROR Error when sending message to topic partition_change1 with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [partition_change1]


I am facing this issue for quit sometime, i need to fix this as i have some business impact. ASAP.

Highlighted

Re: kafka send message error

New Contributor

@sunil thupakula

kafka server logs:

Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread) [2018-05-24 10:11:55,890] INFO [ReplicaFetcher replicaId=1004, leaderId=1003, fetcherId=0] Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread)

Any idea, why i am not getting cluster authorization to kafka topics ?

Highlighted

Re: kafka send message error

New Contributor

@sunil thupakula

More Logs :

Kafka Server Logs:

Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread) [2018-05-24 10:11:55,890] INFO [ReplicaFetcher replicaId=1004, leaderId=1003, fetcherId=0] Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread)