Support Questions

Find answers, ask questions, and share your expertise

How to choose number of kafka partitions


Right now we have only one topic , with 3 partitions and with a replicaiton factor of 3. My log.dirs has 4 mount point or paths or directories.

Only 1 mount point is being used out of 4 . Other 3 mount point or paths or directories , mentioned in log.dirs are not been used.

How can i make all the mount / directories , get used.





@Jacqualin jasmin

Can you locate the kafka ? The log dir parameter which is a comma separeted should list of directories is where kafka stores the log files. see example below of 3 different log dirs


Ensure that the directories you configure are listed and restart kafka.


Yes my log.dirs has 4 comma separated directories. Out of 4, only one directory is getting used. How can we make sure all the 4 directories are equally be used.

Right now I have 3 brokers and my topic has 3 partitions. 3 partitions are spread across 3 brokers.

my question is why all the directories were not getting filled ( why only one ). Does the number of partitions of a topic should depend on the number of directories in log.dirs?



Expert Contributor

@Jacqualin jasmin - what is your partitioning scheme? Keep in mind:

The default partitioning strategy has following flow while deciding partition,
  • If a partition is specified in the record, use it
  • If no partition is specified but a key is present choose a partition based on a hash of the key
  • If no partition or key is present choose a partition in a round-robin fashion

(that is from this HCC article)

Is it possible you're pushing records through one partition?

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.