Community Articles

Find and share helpful community-sourced technical articles.
Announcements
Celebrating as our community reaches 100,000 members! Thank you!
Labels (1)
avatar
Master Guru

A short how-to article on how to list CDP Data hub Kafka topics via command line:

Prerequisites

  • CDP DataHub Kafka cluster
  • Workload username & password
  • CDP PC trust store

Steps

  1. Fetch CDP cert and store in a trust store

 

 

/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks

 

 

  • ssh into any Data hub Kafka node using CDP workload username/password
  • Copy trust store onto the node
  • Set broker host

 

 

 

BROKER_HOST=$(hostname -f)

 

 

 

  • Create jaas.conf

 

 

 

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="<workload username>"
password="<workload password>";
};

 

 

 

  • Set KAFKA_OPTS environment variable to point to location where jaas.conf is located

 

 

 

export KAFKA_OPTS=-Djava.security.auth.login.config=<LOCATION-OF-YOUR-JAASCONF>/jaas.conf

 

 

 

  • Create client-kerberos.properties

 

 

 

security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.kerberos.service.name=kafka
ssl.truststore.location=/<LOCATION-OF-YOUR-TRUSTSTORE>/truststore.jks
org.apache.kafka.common.security.plain.PlainLoginModule required username=<WORKLOAD-USER-NAME> password=<WORKLOAD-PASSWORD>

 

 

 

  • Run Kafka-topics

 

 

 

/opt/cloudera/parcels/CDH/bin/kafka-topics --list --bootstrap-server $BROKER_HOST:9093 --command-config <LOCATION-OF-YOUR-CLIENT-KERB-PROP-FILE>/client-kerberos.properties

 

 

 

 

 

 

 

 

 

 

 

 

 

1,662 Views
0 Kudos
Comments
avatar
Super Guru

Some more examples here: https://github.com/asdaraujo/cdp-examples

Version history
Last update:
‎05-24-2022 09:00 PM
Updated by: