List CDP DataHub Kafka Topics using CLI
A short how-to article on how to list CDP Data hub Kafka topics via command line:
Prerequisites
- CDP DataHub Kafka cluster
- Workload username & password
- CDP PC trust store
Steps
- Fetch CDP cert and store in a trust store
- Creating TLS truststore
- or simple fetch it from any node
/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_truststore.jks
- ssh into any Data hub Kafka node using CDP workload username/password
- Copy trust store onto the node
- Set broker host
BROKER_HOST=$(hostname -f)
- Create jaas.conf
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username="<workload username>"
password="<workload password>";
};
- Set KAFKA_OPTS environment variable to point to location where jaas.conf is located
export KAFKA_OPTS=-Djava.security.auth.login.config=<LOCATION-OF-YOUR-JAASCONF>/jaas.conf
- Create client-kerberos.properties
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.kerberos.service.name=kafka
ssl.truststore.location=/<LOCATION-OF-YOUR-TRUSTSTORE>/truststore.jks
org.apache.kafka.common.security.plain.PlainLoginModule required username=<WORKLOAD-USER-NAME> password=<WORKLOAD-PASSWORD>
- Run Kafka-topics
/opt/cloudera/parcels/CDH/bin/kafka-topics --list --bootstrap-server $BROKER_HOST:9093 --command-config <LOCATION-OF-YOUR-CLIENT-KERB-PROP-FILE>/client-kerberos.properties
