Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

kafka send message error

Solved Go to solution

kafka send message error

Contributor

After enable ranger kafka plugin, execute command "/usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list bigdata001:6667 --topic test5", but when I input content to send message, the result returns as follows:

[2017-03-30 17:06:45,507] WARN Error while fetching metadata with correlation id 0 : {test5=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2017-03-30 17:06:45,507] ERROR Error when sending message to topic test5 with key: null, value: 7 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [test5]
[2017-03-30 17:11:45,563] WARN Error while fetching metadata with correlation id 1 : {test5=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: kafka send message error

Contributor
8 REPLIES 8

Re: kafka send message error

Super Mentor

@Zhao Chaofeng

Please login to Ranger UI http://RANGER_HOST:6080/index.html#!/policymanager/resource

Check the policies defined for Kafka (specially for topics) that you are not restricting access.

.

Purely at Kafka level you can check the permissions using the following utility:

# bin/kafka-acls.sh --list --topic <TOPIC_NAME>

.

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_secure-kafka-ambari/content/kafka-acl-exa...

Re: kafka send message error

Contributor

Yes, I defined a policy in ranger as follows:

14217-selection-049.png

Re: kafka send message error

Contributor
# bin/kafka-acls.sh --list --topic test5

After executing the command, there is no acls for topic test5.

Highlighted

Re: kafka send message error

Contributor

Re: kafka send message error

Contributor

OK, thank you for your reply.

Re: kafka send message error

New Contributor

@sunil thupakulaI

i am also getting the same error when i try to publish/consume messages.

I have created a topic with one partition, using kafka user , though i have 3 brokers in my kafka cluster, i have also authorized using ranger policy . When ever i alter the topic partitions to more than 1 then i get below error.


ERROR Error when sending message to topic partition_change1 with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [partition_change1]


I am facing this issue for quit sometime, i need to fix this as i have some business impact. ASAP.

Re: kafka send message error

New Contributor

@sunil thupakula

kafka server logs:

Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread) [2018-05-24 10:11:55,890] INFO [ReplicaFetcher replicaId=1004, leaderId=1003, fetcherId=0] Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread)

Any idea, why i am not getting cluster authorization to kafka topics ?

Re: kafka send message error

New Contributor

@sunil thupakula

More Logs :

Kafka Server Logs:

Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread) [2018-05-24 10:11:55,890] INFO [ReplicaFetcher replicaId=1004, leaderId=1003, fetcherId=0] Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread)

Don't have an account?
Coming from Hortonworks? Activate your account here