Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka SSL - console-consumer on node5 not receiving messages from producer on node4

Kafka SSL - console-consumer on node5 not receiving messages from producer on node4

Expert Contributor

Hi All -

I've Kafka 0.10 .. I've enabled SSL(Non-kerberized) for Kafka Broker on Node 4, and i'm able to produce/consume messages using console-producer & console-consumer from Node 4.

However, when i'm having issues enabling ssl connection between Node 4 & Node 5 & try to consume messages from Node5 (using console-consumer), i'm facing issues.

here are the steps -

On node4 : Started console-producer, publishing on SSL port (9192)

$CONFLUENT/bin/kafka-console-producer --broker-list node4:9192 --topic ssl --producer.config client-ssl.properties 

Started console-consumer on node4, consuming messages from console producer (port 9192)

$CONFLUENT/bin/kafka-console-consumer --bootstrap-server node4:9192 --topic ssl --new-consumer --consumer.config client-ssl.properties

contents of client-ssl.properties ---------------------------------

security.protocol=SSL 
ssl.truststore.location=/usr/hdp/2.5.3.0-37/confluent-3.2.2/kafkaSSL/kafka.client.truststore.jks 
ssl.truststore.password=<passwd> 
ssl.keystore.location=/usr/hdp/2.5.3.0-37/confluent-3.2.2/kafkaSSL/kafka.client.keystore.jks 
ssl.keystore.password=<passwd> 
ssl.key.password=<passwd> 

This is working fine, and consumer is able to consumer messages produced by the producer.

On Node5, i've another Kafka instance, i start another console-consumer from Node5, for consuming the data from console-producer in node4

To enable SSL (between client on Node5 & Broker on Node4),

Steps on Node 4 ->

1) create kafka.client05.keystore.jks (set the CN -> Node5)

keytool -keystore kafka.client05.keystore.jks -alias localhost -validity 365 -genkey

- Export certificate from keystore

keytool -keystore kafka.client05.keystore.jks -alias localhost -certreq -file cert05-file 

- Get certificate signed by CA

openssl x509 -req -CA ca-cert -CAkey ca-key -in cert05-file -out cert05-signed -days 365 -CAcreateserial -passin pass:<passwd> 

- Import the CA Certificate & Signed certificate into the keystore

keytool -keystore kafka.client05.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore kafka.client05.keystore.jks -alias localhost -import -file cert05-signed 

Add the generated CA to the client's truststore

keytool -keystore kafka.client05.truststore.jks -alias CARoot -import -file ca-cert 

On Node5, i do the same steps above,and start the console-consumer (to read data produced by console-producer on node4)

$CONFLUENT1/bin/kafka-console-consumer --bootstrap-server node4:9192 --topic ssl --new-consumer --consumer.config client-ssl.properties 

This is not working.. what do i need to make this work ?

2 REPLIES 2
Highlighted

Re: Kafka SSL - console-consumer on node5 not receiving messages from producer on node4

Expert Contributor

@mqureshi, @Terry Stebbens - any ideas on this ?

Re: Kafka SSL - console-consumer on node5 not receiving messages from producer on node4

Super Guru
@Karan Alang

One thing that jumps out here is that you are using "-alias localhost". this cannot be the case when you are communicating between two physically different servers. Give the right DNS name for node04 and node05. This name should be the same using which you should be successfully able to "ping <node04/05>" and get a reply.