question Re: cant create a Kafka topic in Archives of Support Questions (Read Only)

cant create a Kafka topic

aliyesami — Fri, 13 May 2016 01:16:42 GMT

when running the command "

kafka-topics.sh --create --zookeeper ip-172-31-8-79.ec2.internal --replication-factor 1 --partitions 4 --topic mytopic"

I am getting the following msgs , and it exits after couple of secs. Its saying it cant connect to zookeeper but zookeeper is up and running .

[2016-05-12 12:32:10,272] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn) java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)

at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) Exception in thread "main" org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 30000

Re: cant create a Kafka topic

aliyesami — Fri, 13 May 2016 01:16:43 GMT

Telnet showing connection refused to port 2181 eventhough zookeeper is running :

[hdfs@ip-172-31-8-79 ~]$ telnet ip-172-31-8-79.ec2.internal 2181 Trying 172.31.8.79... telnet: connect to address 172.31.8.79: Connection refused [hdfs@ip-172-31-8-79 ~]$ ps -ef | grep "zookeper.properties" hdfs 22894 22772 0 12:52 pts/1 00:00:00 grep zookeper.properties [hdfs@ip-172-31-8-79 ~]$

Re: cant create a Kafka topic

aliyesami — Fri, 13 May 2016 01:16:43 GMT

I see there is no zookeeper server running on this node but only zookeeper client . this node is called Slave1 , all three zookeeper server processes are running on MasterNode. and telnet from Slave1 to MasterNode on port 2181 works fine. so why I am getting connection refused ? I am running the kafka command on Slave1.

[hdfs@ip-172-31-8-79 ~]$ telnet MasterNode 2181 Trying 172.31.8.77... Connected to MasterNode. Escape character is '^]'.

Re: cant create a Kafka topic

aliyesami — Fri, 13 May 2016 01:16:43 GMT

this step also did not help :

put maxClientCnxns=30 in file /etc/zookeeper/conf/zoo.cfg

retart zookeeper :

Re: cant create a Kafka topic

aliyesami — Fri, 13 May 2016 01:16:43 GMT

actually zookeeper server is running on three different servers ,namely Slave3, Master, SecondaryMaster

Re: cant create a Kafka topic

pshah — Fri, 13 May 2016 01:16:43 GMT

Can you try putting the port in the --zookeeper option. Like ip-172-31-8-79.ec2.internal:2181 ?

Re: cant create a Kafka topic

aliyesami — Fri, 13 May 2016 01:23:29 GMT

ah dumb me I was not using the right host address .

[hdfs@ip-172-31-8-79 ~]$ kafka-topics.sh --create --zookeeper MasterNode --replication-factor 1 --partitions 4 --topic mytopic Created topic "mytopic". [hdfs@ip-172-31-8-79 ~]$

Re: cant create a Kafka topic

rkandula — Tue, 22 Nov 2016 04:27:57 GMT

run the following command before creating topic

export KAFKA_CLIENT_KERBEROS_PARAMS="-Djava.security.auth.login.config=/usr/hdp/current/kafka-broker/config/kafka_client_jaas.conf"