Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

getting an issue in secure Kafka with SSL certs

Highlighted

getting an issue in secure Kafka with SSL certs

New Contributor

42930-capture-111.png

42931-112.png

42932-113.png

42933-114.png

42934-115.png

Getting issue in SSL for KAFKA CLUSTER

I have created a three node cluster of kafka, and one ambari metrics node, total four nodes.

Created certificates for all the nodes and created keystore and truststore for all the nodes with appropriate certificates.

All the certs are created by openssl

listeners=PLAINTEXT://:6668,SSL://:6667

Added some of the addition parameter in ambari for ssl,

ssl.keystore.location=/opt/ssl/ssl-cert.keystore

ssl.keystore.password=changeit

ssl.key.password=changeit

ssl.truststore.location=/opt/ssl/ssl-cert.truststore

ssl.truststore.password=changeit

ssl.endpoint.identification.algorithm=HTTPS

ssl.keystore.type=JKS

ssl.truststore.type=JKS

ssl.client.auth=required

security.inter.broker.protocol=SSL

ssl.secure.random.implementation=SHA1PRNG

I have configured the ssl protocol, keystore, truststore in nifi as well.

Both the nifi and kafka certs are signed by same CA (certificate authority)

So when I am trying to push the data form nifi to kafka its not flowing, I am getting an error in start the consumer to check the data flow.

/usr/hdf/current/kafka-broker/bin/kafka-console-consumer.sh --bootstp-server node1.com:6667,node2.com:6667,node3..com:6667 --topic datacheck2

I have tried the command to check whether an error in ssl certs, but it seems to be good.

openssl s_client -debug -connect localhost:6667 -tls1

2 REPLIES 2
Highlighted

Re: getting an issue in secure Kafka with SSL certs

Super Mentor

@ashok kumar

Can you try increasing the Kafka Heap memory to a bit more and then try again?

# export KAFKA_HEAP_OPTS="-Xmx4g -Xms4g" 
# /usr/hdf/current/kafka-broker/bin/kafka-console-consumer.sh --bootstp-server node1.kafka.dev.tt-dp.com:6667,node2.kafka.dev.tt-dp.com:6667,node3.kafka.dev.tt-dp.com:6667 --topic datacheck2

.

Or try increating the heap memory inside the "kafka-env.sh" script.

Highlighted

Re: getting an issue in secure Kafka with SSL certs

New Contributor

I tried increase the heap size, i created a topic datachaeck2, but i am trying to send message form producer to consumer in secure mode(SSL) its not working.

/usr/hdf/current/kafka-broker/bin/kafka-console-consumer.sh --bootstrap-server node1.kafka:6667,node2.kafka:6667,node3.kafka:6667 --topic datacheck2

/usr/hdf/current/kafka-broker/bin/kafka-console-producer.sh -broker-list node1.kafka:6667,node2.kafka:6667,node3.kafka:6667 --topic datacheck2

but i am able to send message from producer to consumer in unsecure mode. (PLAINTEXT)

/usr/hdf/current/kafka-broker/bin/kafka-console-consumer.sh --bootstrap-server node1.kafka:6668,node2.kafka:6668,node3.kafka:6668 --topic datacheck2

usr/hdf/current/kafka-broker/bin/kafka-console-producer.sh -broker-list node1.kafka:6668,node2.kafka:6668,node3.kafka:6668 --topic datacheck2

Don't have an account?
Coming from Hortonworks? Activate your account here