Creating a topic in kakfa fails with "ERROR kafka.admin.AdminOperationException: replication factor: 2 larger than available brokers: 0"

[#1]

 HDP version : 2.5

Here are the steps which I have followed :

1. Installed kafka brokers on 2 nodes in my cluster.

2. Tried to create a topic , Created successfully

3. I tried to change the zookeeper root directory for kafka by following below steps :

> In ambari > Kafka > Configs > changed value of zookeeper.connect as follows :

node1.example.com:2181,node2.example.com:2181,node1.example.com:2181/kafka

4. Saved the changes and manually created /kafka in zookeeper using zkCli.sh

5. Restarted kafka brokers. After restart znodes were created in /kafka in zookeeper.

6. Now when I try to create a topic it fails with below stack trace :

Error while executing topic command : replication factor: 2 larger than available brokers: 0
[2016-12-02 16:13:12,851] ERROR kafka.admin.AdminOperationException: replication factor: 2 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:117)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:403)
at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:110)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:61)
at kafka.admin.TopicCommand.main(TopicCommand.scala)

I can confirm that all znodes are created correctly under /kafka, brokers are up and running fine. I suspect that for some reason kafka admin or kafka controller(not sure who looks at the brokers znode value) is not able to identify live brokers because it tries to find the broker ids in previous zookeeper root dir i.e /brokers/ids/ which is empty.

In order to confirm if this is the scenario, I reverted the changes which were made.

 > Changed the zookeeper root directory to default i.e <node1.example.com:2181,node2.example.com:2181,node1.example.com:2181>

> Restarted zookeeper broker

> Confirmed that all znodes are with correct value (/controller_epoch) (/brokers/ids/1004 /brokers/ids/1006)

Now I tried to create a topic which got created without any issues.

Then I deleted both the broker IDS in /brokers/ids and tried to create a topic. It failed with same error i,e :

Error while executing topic command : replication factor: 2 larger than available brokers: 0 [2016-12-02 16:13:12,851] ERROR

Can some one please help me to understand this behaviour ?

[#2]

@Sriharsha Chintalapani

@Rajkumar Singh

[#3 · Accepted solution]

@amankumbare can you make sure you are passing the zookeeper root as well to your kafka-topics.sh. It should look like this

./bin/kafka-topics.sh --zookeeper

1. node1.example.com:2181,node2.example.com:2181,node1.example.com:2181/kafka --topic name_of_topic --partitions 2 --replication-factor 2 --create

[#4]

@Sriharsha Chintalapani

Yes ! It worked. Thanks a lot.

/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --zookeeper node1.openstacklocal:2181/kafka  --create  --topic my-topic  --partitions 2  --replication-factor 2 
Created topic "my-topic".