Support Questions

yjiang · ‎01-22-2017

Hi,

After enabled Kerberos using Ambari, I got problem creating topics in Kafka using the kafka-topics.sh script. The topic was created, but its status is wrong without leader. It seems the topic is created with PLAINTEXT, while there is only PLAINTEXTSASL broker in the cluster after enabled Kerberos. The only configuration change I made is to chagne broker listener from 'PLAINTEXT://localhost:6667' to 'PLAINTEXTSASL://localhost:6667'. As posted in this question, I also changed the kafka-topics.sh to make it work with Kerberos. I am using HDP2.5.3.

$ ./kafka-topics.sh --zookeeper ip-10-0-0-149.ap-northeast-1.compute.internal --create --partitions 1 --replication-factor 1 --topic mytopic
Created topic "mytopic".

$ ./kafka-topics.sh --zookeeper ip-10-0-0-149.ap-northeast-1.compute.internal --describe --topic mytopic
Topic:mytopic	PartitionCount:1	ReplicationFactor:1	Configs:
	Topic: mytopic	Partition: 0	Leader: none	Replicas: 1001	Isr:

mthiele1 · ‎01-23-2017

When you use a script, command, or API to create a topic, an entry is created under ZooKeeper. The only user with access to ZooKeeper is the service account running Kafka (by default, kafka). Therefore, the first step toward creating a Kafka topic on a secure cluster is to run kinit, specifying the Kafka service keytab. The second step is to create the topic.

Run kinit, specifying the Kafka service keytab. For example:
kinit -k -t /etc/security/keytabs/kafka.service.keytab kafka/c6401.ambari.apache.org@EXAMPLE.COM
Next, create the topic. Run the kafka-topics.sh command-line tool with the following options:
/bin/kafka-topics.sh --zookeeper <hostname>:<port> --create --topic <topic-name> --partitions <number-of-partitions> --replication-factor <number-of-replicating-servers>
For example:
```
/bin/kafka-topics.sh --zookeeper c6401.ambari.apache.org:2181 --create --topic test_topic --partitions 2 --replication-factor 2

Created topic "test_topic".
```

View solution in original post

amankumbare · ‎03-07-2017

@yjiang

If you try to create a topic as a non kafka user, it creates a topic but with no Leader and ISR. This is a known issue. According to me, the reason behind this could be the zookeeper acl's. Once topic is created in zookeeper, its acl's will not allow kafka to read details about it.

If you want to create a topic as a non kafka user you need to workaround by following below steps :

If you are not using Ranger :

1. Make sure "auto.create.topic.enable = true"

2. Give acl's for the user from which you want to create a topic, for ex :

# bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:Bob --producer --topic Test-topic

3. Do a kinit as a user from which you want to create topic.

4. Now try to produce messages to topic as that user :

# ./kafka-console-producer.sh --broker-list <hostname-broker>:6667 --topic Test-topic --security-protocol PLAINTEXTSASL

If you are using Ranger :

Instead of point 2 in above steps you will need to add a policy for the topic in ranger. Allow permissions for that user to produce, create, consumer. Restart kafka service. Then follow step 3 and 4 as mentioned above.

Hope this helps !!

Cloudera Community

Support Questions

Problem of Creating Topics in Kafka with Kerberos