Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

kafka broker is not starting after configuring it for encryption with SASL_SSL protocol

Solved Go to solution

kafka broker is not starting after configuring it for encryption with SASL_SSL protocol

New Contributor

[2018-09-11 00:06:58,404] INFO Successfully logged in. (org.apache.kafka.common.security.authenticator.AbstractLogin) [2018-09-11 00:06:58,409] INFO [Principal=kafka/kafka1.example.com@EXAMPLE.COM]: TGT refresh thread started. (org.apache.kafka.common.security.kerberos.KerberosLogin) [2018-09-11 00:06:58,409] INFO [Principal=kafka/kafka1.example.com@EXAMPLE.COM]: TGT valid starting at: Tue Sep 11 00:06:58 EDT 2018 (org.apache.kafka.common.security.kerberos.KerberosLogin) [2018-09-11 00:06:58,409] INFO [Principal=kafka/kafka1.example.com@EXAMPLE.COM]: TGT expires: Tue Sep 11 10:06:58 EDT 2018 (org.apache.kafka.common.security.kerberos.KerberosLogin) [2018-09-11 00:06:58,410] INFO [Principal=kafka/kafka1.example.com@EXAMPLE.COM]: TGT refresh sleeping until: Tue Sep 11 08:23:38 EDT 2018 (org.apache.kafka.common.security.kerberos.KerberosLogin) [2018-09-11 00:06:58,411] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: SSL trust store is specified, but trust store password is not specified. at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93) at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:63) at kafka.network.Processor.<init>(SocketServer.scala:422) at kafka.network.SocketServer.newProcessor(SocketServer.scala:155) at kafka.network.SocketServer.$anonfun$startup$2(SocketServer.scala:96) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:156) at kafka.network.SocketServer.$anonfun$startup$1(SocketServer.scala:95) at kafka.network.SocketServer.$anonfun$startup$1$adapted(SocketServer.scala:90) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at kafka.network.SocketServer.startup(SocketServer.scala:90) at kafka.server.KafkaServer.startup(KafkaServer.scala:215) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39) at kafka.Kafka$.main(Kafka.scala:67) at kafka.Kafka.main(Kafka.scala) Caused by: org.apache.kafka.common.KafkaException: SSL trust store is specified, but trust store password is not specified. at org.apache.kafka.common.security.ssl.SslFactory.createTruststore(SslFactory.java:195) at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:115) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:91) ... 16 more [2018-09-11 00:06:58,416] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer) [2018-09-11 00:06:58,421] INFO [Socket Server on Broker 1], Shutting down (kafka.network.SocketServer) [2018-09-11 00:06:58,426] WARN (kafka.utils.CoreUtils$) java.lang.NullPointerException at kafka.network.SocketServer.$anonfun$shutdown$3(SocketServer.scala:129) at kafka.network.SocketServer.$anonfun$shutdown$3$adapted(SocketServer.scala:129) at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:32) at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:29) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:193) at kafka.network.SocketServer.shutdown(SocketServer.scala:129) at kafka.server.KafkaServer.$anonfun$shutdown$3(KafkaServer.scala:582) at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:78) at kafka.utils.Logging.swallowWarn(Logging.scala:94) at kafka.utils.Logging.swallowWarn$(Logging.scala:93) at kafka.utils.CoreUtils$.swallowWarn(CoreUtils.scala:48) at kafka.utils.Logging.swallow(Logging.scala:96) at kafka.utils.Logging.swallow$(Logging.scala:96) at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:48) at kafka.server.KafkaServer.shutdown(KafkaServer.scala:582) at kafka.server.KafkaServer.startup(KafkaServer.scala:289) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39) at kafka.Kafka$.main(Kafka.scala:67) at kafka.Kafka.main(Kafka.scala) [2018-09-11 00:06:58,433] INFO Shutting down. (kafka.log.LogManager) [2018-09-11 00:06:58,448] INFO Shutdown complete. (kafka.log.LogManager) [2018-09-11 00:06:58,448] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread) [2018-09-11 00:06:58,555] INFO Session: 0x165b374ac140029 closed (org.apache.zookeeper.ZooKeeper) [2018-09-11 00:06:58,555] INFO EventThread shut down for session: 0x165b374ac140029 (org.apache.zookeeper.ClientCnxn) [2018-09-11 00:06:58,562] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer) [2018-09-11 00:06:58,564] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable) org.apache.kafka.common.KafkaException: org.apache.kafka.common.KafkaException: SSL trust store is specified, but trust store password is not specified. at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94) at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93) at org.apache.kafka.common.network.ChannelBuilders.serverChannelBuilder(ChannelBuilders.java:63) at kafka.network.Processor.<init>(SocketServer.scala:422) at kafka.network.SocketServer.newProcessor(SocketServer.scala:155) at kafka.network.SocketServer.$anonfun$startup$2(SocketServer.scala:96) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:156) at kafka.network.SocketServer.$anonfun$startup$1(SocketServer.scala:95) at kafka.network.SocketServer.$anonfun$startup$1$adapted(SocketServer.scala:90) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at kafka.network.SocketServer.startup(SocketServer.scala:90) at kafka.server.KafkaServer.startup(KafkaServer.scala:215) at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39) at kafka.Kafka$.main(Kafka.scala:67) at kafka.Kafka.main(Kafka.scala) Caused by: org.apache.kafka.common.KafkaException: SSL trust store is specified, but trust store password is not specified. at org.apache.kafka.common.security.ssl.SslFactory.createTruststore(SslFactory.java:195) at org.apache.kafka.common.security.ssl.SslFactory.configure(SslFactory.java:115) at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:91) ... 16 more

1 ACCEPTED SOLUTION

Accepted Solutions

Re: kafka broker is not starting after configuring it for encryption with SASL_SSL protocol

Super Mentor

@Ankita Ghate
We see the error cause as following:

Caused by: org.apache.kafka.common.KafkaException: SSL trust store is specified, but trust store password is not specified.


We see that you are getting the above error in your Kafka. Which indicates that while configuring truststore for Kafka you might have forgotten to add the "ssl.truststore.password" property properly. Can you please check your Kafka configs to see if you have setup the truststore properly as mentioned in Reference Doc: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_security/content/ch_wire-kafka.html

ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password = test1234
ssl.key.password = test1234
ssl.truststore.location = /var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password = test1234

Values may be based on your requirement but need to make sure that it has correctly defined ssl.truststore.password

.

2 REPLIES 2

Re: kafka broker is not starting after configuring it for encryption with SASL_SSL protocol

Super Mentor

@Ankita Ghate
We see the error cause as following:

Caused by: org.apache.kafka.common.KafkaException: SSL trust store is specified, but trust store password is not specified.


We see that you are getting the above error in your Kafka. Which indicates that while configuring truststore for Kafka you might have forgotten to add the "ssl.truststore.password" property properly. Can you please check your Kafka configs to see if you have setup the truststore properly as mentioned in Reference Doc: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_security/content/ch_wire-kafka.html

ssl.keystore.location = /var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password = test1234
ssl.key.password = test1234
ssl.truststore.location = /var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password = test1234

Values may be based on your requirement but need to make sure that it has correctly defined ssl.truststore.password

.

Re: kafka broker is not starting after configuring it for encryption with SASL_SSL protocol

New Contributor

@Jay Kumar SenSharma thanks for the response.

I had provided ssl.truststore.password.generator but not ssl.truststore.password. Now I have added ssl.truststore.password kafka has started but not able to produce messages, giving error as below,

[2018-09-11 01:21:52,015] ERROR Error when sending message to topic test with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. [2018-09-11 01:22:52,020] ERROR Error when sending message to topic test with key: null, value: 1 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.

Don't have an account?
Coming from Hortonworks? Activate your account here