HI,
I am using Cloudera CDH 5.14.4
I have added csd file and then downloaded and actiavted kafka parcel. After adding the kafka service I am unable to start the kafka brokers.
csd: kafka-1.2.0.jar
Kafka parcel: 2.0.2-1.2.0.2.p0.5
I found below error in logfile /var/log/kafka/kafka-broker-hostnameXXX.log
2018-08-22 16:27:07,747 INFO kafka.server.KafkaServer: starting
2018-08-22 16:27:07,754 INFO kafka.server.KafkaServer: Connecting to zookeeper on b2brp-cdh-cedn0.hostanameXXXXXX:2181,b2brp-cdh-cedn1.hostanameXXXXXX:2181,b2brp-cdh-cmsn0.hostanameXXXXXX:2181,b2brp-cdh-cmsn1.hostanameXXXXXX:2181,b2brp-cdh-cmsn2.hostanameXXXXXX:2181
2018-08-22 16:27:07,755 INFO org.I0Itec.zkclient.ZkClient: JAAS File name: /run/cloudera-scm-agent/process/1160-kafka-KAFKA_BROKER/jaas.conf
2018-08-22 16:27:07,757 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=b2brp-cdh-cedn0.hostanameXXXXXX:2181,b2brp-cdh-cedn1.hostanameXXXXXX:2181,b2brp-cdh-cmsn0.hostanameXXXXXX:2181,b2brp-cdh-cmsn1.hostanameXXXXXX:2181,b2brp-cdh-cmsn2.hostanameXXXXXX:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@7a220c9a
2018-08-22 16:27:07,757 INFO org.I0Itec.zkclient.ZkEventThread: Starting ZkClient event thread.
2018-08-22 16:27:07,759 INFO org.I0Itec.zkclient.ZkClient: Waiting for keeper state SaslAuthenticated
2018-08-22 16:27:07,764 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
2018-08-22 16:27:07,765 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server b2brp-cdh-cmsn1.hostanameXXXXXX/xx.xx.xx.xx:2181. Will attempt to SASL-authenticate using Login Context section 'Client'
2018-08-22 16:27:07,766 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to b2brp-cdh-cmsn1.hostanameXXXXXX/xx.xx.xx.xx:2181, initiating session
2018-08-22 16:27:07,776 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server b2brp-cdh-cmsn1.hostanameXXXXXX/xx.xx.xx.xx:2181, sessionid = 0x564e90a3c9c9b85, negotiated timeout = 6000
2018-08-22 16:27:07,776 INFO org.I0Itec.zkclient.ZkClient: zookeeper state changed (SyncConnected)
2018-08-22 16:27:07,784 INFO org.I0Itec.zkclient.ZkClient: zookeeper state changed (SaslAuthenticated)
2018-08-22 16:27:07,830 INFO kafka.log.LogManager: Loading logs.
2018-08-22 16:27:07,837 INFO kafka.log.LogManager: Logs loading complete.
2018-08-22 16:27:08,108 INFO kafka.log.LogManager: Starting log cleanup with a period of 300000 ms.
2018-08-22 16:27:08,110 INFO kafka.log.LogManager: Starting log flusher with a default period of 9223372036854775807 ms.
2018-08-22 16:27:08,112 INFO kafka.log.LogCleaner: Starting the log cleaner
2018-08-22 16:27:08,117 INFO kafka.log.LogCleaner: [kafka-log-cleaner-thread-0], Starting
2018-08-22 16:27:08,124 FATAL kafka.server.KafkaServer: Fatal error during KafkaServer startup. Prepare to shutdown
kafka.common.InconsistentBrokerIdException: Configured broker.id 341 doesn't match stored broker.id 186 in
meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to
create a new broker, you should remove all data in your data directories (log.dirs).
at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:635)
at kafka.server.KafkaServer.startup(KafkaServer.scala:184)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
at kafka.Kafka$.main(Kafka.scala:67)
at com.cloudera.kafka.wrap.Kafka$.main(Kafka.scala:76)
at com.cloudera.kafka.wrap.Kafka.main(Kafka.scala)
2018-08-22 16:27:08,126 INFO kafka.server.KafkaServer: shutting down
2018-08-22 16:27:08,129 INFO kafka.log.LogManager: Shutting down.
2018-08-22 16:27:08,130 INFO kafka.log.LogCleaner: Shutting down the log cleaner.
2018-08-22 16:27:08,130 INFO kafka.log.LogCleaner: [kafka-log-cleaner-thread-0], Shutting down
2018-08-22 16:27:08,131 INFO kafka.log.LogCleaner: [kafka-log-cleaner-thread-0], Stopped
2018-08-22 16:27:08,131 INFO kafka.log.LogCleaner: [kafka-log-cleaner-thread-0], Shutdown completed
2018-08-22 16:27:08,137 INFO kafka.log.LogManager: Shutdown complete.
2018-08-22 16:27:08,138 INFO org.I0Itec.zkclient.ZkEventThread: Terminate ZkClient event thread.
2018-08-22 16:27:08,147 INFO org.apache.zookeeper.ZooKeeper: Session: 0x564e90a3c9c9b85 closed
2018-08-22 16:27:08,147 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2018-08-22 16:27:08,149 INFO kafka.server.KafkaServer: shut down completed
2018-08-22 16:27:08,150 FATAL kafka.server.KafkaServerStartable: Fatal error during KafkaServerStartable startup. Prepare to shutdown
kafka.common.InconsistentBrokerIdException: Configured broker.id 341 doesn't match stored broker.id 186 in meta.properties. If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs).
at kafka.server.KafkaServer.getBrokerId(KafkaServer.scala:635)
at kafka.server.KafkaServer.startup(KafkaServer.scala:184)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
at kafka.Kafka$.main(Kafka.scala:67)
at com.cloudera.kafka.wrap.Kafka$.main(Kafka.scala:76)
at com.cloudera.kafka.wrap.Kafka.main(Kafka.scala)
2018-08-22 16:27:08,150 INFO kafka.server.KafkaServer: shutting down
Can anyone please help me to resolve this issue.
Thanks in advance.
