Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

kafka send message error

avatar
Contributor

After enable ranger kafka plugin, execute command "/usr/hdp/current/kafka-broker/bin/kafka-console-producer.sh --broker-list bigdata001:6667 --topic test5", but when I input content to send message, the result returns as follows:

[2017-03-30 17:06:45,507] WARN Error while fetching metadata with correlation id 0 : {test5=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)
[2017-03-30 17:06:45,507] ERROR Error when sending message to topic test5 with key: null, value: 7 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [test5]
[2017-03-30 17:11:45,563] WARN Error while fetching metadata with correlation id 1 : {test5=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient
1 ACCEPTED SOLUTION
9 REPLIES 9

avatar
Master Mentor

@Zhao Chaofeng

Please login to Ranger UI http://RANGER_HOST:6080/index.html#!/policymanager/resource

Check the policies defined for Kafka (specially for topics) that you are not restricting access.

.

Purely at Kafka level you can check the permissions using the following utility:

# bin/kafka-acls.sh --list --topic <TOPIC_NAME>

.

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_secure-kafka-ambari/content/kafka-acl-exa...

avatar
Contributor

Yes, I defined a policy in ranger as follows:

14217-selection-049.png

avatar
Contributor
# bin/kafka-acls.sh --list --topic test5

After executing the command, there is no acls for topic test5.

avatar
Contributor

OK, thank you for your reply.

avatar
New Contributor

@sunil thupakulaI

i am also getting the same error when i try to publish/consume messages.

I have created a topic with one partition, using kafka user , though i have 3 brokers in my kafka cluster, i have also authorized using ranger policy . When ever i alter the topic partitions to more than 1 then i get below error.


ERROR Error when sending message to topic partition_change1 with key: null, value: 4 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [partition_change1]


I am facing this issue for quit sometime, i need to fix this as i have some business impact. ASAP.

avatar
New Contributor

@sunil thupakula

kafka server logs:

Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread) [2018-05-24 10:11:55,890] INFO [ReplicaFetcher replicaId=1004, leaderId=1003, fetcherId=0] Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread)

Any idea, why i am not getting cluster authorization to kafka topics ?

avatar
New Contributor

@sunil thupakula

More Logs :

Kafka Server Logs:

Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread) [2018-05-24 10:11:55,890] INFO [ReplicaFetcher replicaId=1004, leaderId=1003, fetcherId=0] Retrying leaderEpoch request for partition hello-4 as the leader reported an error: CLUSTER_AUTHORIZATION_FAILED (kafka.server.ReplicaFetcherThread)

avatar

Make Sure that HDFS Service checkbox is checked  in ClouderaManager>Kafka>Configuration

For me after i checked HDFS Service it was resolved