Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Authorization Error in Kafka Producer(non-kerberized custer)

Highlighted

Authorization Error in Kafka Producer(non-kerberized custer)

Expert Contributor

Hi All,

I've enabled Kafka-Ranger plugin, and then created the command line Kafka producer,

when i publish a message, it gives following error.. seems it is expecting kerberos authentication ?

The Cluster is not kerberized, pls let me know what needs to be done to fix the error.

Command :

$KAFKA_HOME/bin/kafka-console-producer.sh --broker-list sandbox.hortonworks.com:6667 --topic kafka1_topic

ERROR :

[2016-11-20 23:44:27,168] WARN Error while fetching metadata [{TopicMetadata for topic kafka1_topic -> No partition metadata for topic kafka1_topic due to kafka.common.TopicAuthorizationException}] for topic [kafka1_topic]: class kafka.common.TopicAuthorizationException (kafka.producer.BrokerPartitionInfo) [2016-11-20 23:44:27,170] ERROR Failed to send requests for topics kafka1_topic with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler)

8 REPLIES 8

Re: Authorization Error in Kafka Producer(non-kerberized custer)

@Karan Alang

Did you kinit as a user who has been granted publish privileges on the kafka1_topic topic?

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

Expert Contributor

Hi @emaxwell - kerberos is disabled on the cluster .. do i still need to do kinit ?

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

Expert Contributor

@emaxwell, i've kerberized the cluster now, and trying to run kinit for user - kafka1, but getting an error -

Connection to sandbox.hortonworks.com closed. [kafka1@sandbox ~]$ ls -lrt /etc/security/keytabs/kafka.service.keytab -r-------- 1 kafka hadoop 418 2016-11-21 03:51 /etc/security/keytabs/kafka.service.keytab [kafka1@sandbox ~]$ kinit -k -t /etc/security/keytabs/kafka.service.keytab admin/admin@EXAMPLE.COM kinit: Permission denied while getting initial credentials

Do I need to give access to

/etc/security/keytabs/kafka.service.keytab to user kafka1 ?

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

@Karan Alang

I see that you have kerberized the cluster now and trying to do kinit and it is failing..

The reason for this failure is the prinicpal that you have given in the kinit command. Please check the principal for the keytab by executing the below command.

klist -kt /etc/security/keytabs/kafka.service.keytab

This will give you the principal name and then try the kinit command.

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

@Karan Alang

Again your kinit command is wrong. you should use kafka in prinicpal, not kafka1.

Below is the right command.

 kinit -k -t /etc/security/keytabs/kafka.service.keytab kafka1/sandbox.hortonworks.com@EXAMPLE.COM

Are these users(kafka1 & kafka2) created by you? If yes, before executing the above command, you have to have users(kafka1 & kafka2) registered with the kdc. Have you done that?

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

@Karan Alang Seems like 'kafka1' is a unix user and your mit kdc does not know about it. So you have register user with kdc and create keytab for the user. Please follow the below steps.

1. Run kadmin.local

2. In the kadmin prompt, run this command 'addprinc -randkey kafka1'

3. Then this command: 'xst -k /tmp/keytabs/kafka1.headless.keytab kafka1'

This should actually create the keytab and register user kafka1 in kdc.

Now you can do the kinit with the kafka1 keytab(from step2) and perform the operations.

If you have ranger configured in your cluster for kafka, you need to add appropriate policies in ranger.

Hope this helps.

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

@Karan Alang If this is working for you, accept this answer and close the loop.

Highlighted

Re: Authorization Error in Kafka Producer(non-kerberized custer)

Expert Contributor

@Ayub Khan,

this is what i get when i run the above command

[root@sandbox conf]# klist -kt /etc/security/keytabs/kafka.service.keytab Keytab name: FILE:/etc/security/keytabs/kafka.service.keytab KVNO Timestamp Principal ---- ----------------- -------------------------------------------------------- 1 11/21/16 03:51:30 kafka/sandbox.hortonworks.com@EXAMPLE.COM 1 11/21/16 03:51:30 kafka/sandbox.hortonworks.com@EXAMPLE.COM 1 11/21/16 03:51:30 kafka/sandbox.hortonworks.com@EXAMPLE.COM 1 11/21/16 03:51:30 kafka/sandbox.hortonworks.com@EXAMPLE.COM 1 11/21/16 03:51:30 kafka/sandbox.hortonworks.com@EXAMPLE.COM [root@sandbox conf]#

However, when i try to do kinit for users -> kafka1 & kafka2, i get the error shown below ->

[root@sandbox conf]# kinit -k -t /etc/security/keytabs/kafka.service.keytab kafka1/sandbox.hortonworks.com@EXAMPLE.COM kinit: Keytab contains no suitable keys for kafka1/sandbox.hortonworks.com@EXAMPLE.COM while getting initial credentials

What needs to be doe for this ?

Essentially, i want to run Kafka Producer from user - kafka1, & the Kafka consumer from user - kafka2

Don't have an account?
Coming from Hortonworks? Activate your account here