Community Articles
Find and share helpful community-sourced technical articles.
Labels (1)
Rising Star

Quick command to find the total number of partitions in a Kafka cluster, it could help for example in Mirror Maker sizing.
Please replace ZK_SERVER values with your cluster details.

cd /tmp
zookeeper="ZK_SERVER1:2181,ZK_SERVER2:2181,ZK_SERVER3:2181"
sum=0
for i in $(/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --list --zookeeper $zookeeper ); do count=$(/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --describe --zookeeper $zookeeper --topic $i |grep Leader | wc -l); sum=`expr $sum + $count` ;  echo 'total partitions is ' $sum; done

If you want to count partitions with specific filter on the name for Topics

zookeeper="ZK_SERVER1:2181,ZK_SERVER2:2181,ZK_SERVER3:2181"
sum=0 for i in $(/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --list --zookeeper $zookeeper | grep 'FILTER'); do count=$(/usr/hdp/current/kafka-broker/bin/kafka-topics.sh --describe --zookeeper $zookeeper --topic $i |grep Leader | wc -l); sum=`expr $sum + $count` ; echo 'total partitions is ' $sum; done
4,703 Views
0 Kudos
Don't have an account?
Version history
Last update:
‎03-16-2018 02:59 PM
Updated by:
Contributors
Top Kudoed Authors