I use kafka. the server has 1G ethernet
recent i found that then lan bandwidth of machines is too high, i use some tools to monitor the traffic and foud that too many trasfer usage of kafka. total bandwidth frequently reach 400mbps, and ping of other node can from 0.1 ms suddenly to 30ms....
So is there some way to turning the bandwidth usage??
I don't think there is a silver bullet because you have to balance the producers, brokers, and consumers but I would think the most important are these 2 parameters below
batch.size = 16384 # default size
Smaller batch size will make batching less common and may reduce throughput a batch size of zero will disable batching entirely. A very large batch size may use memory a bit more wastefully as we will always allocate a buffer of the specified batch size in anticipation of additional records.
linger.ms = 0 #default
linger.ms=5 would have the effect of reducing the number of requests sent but would add up to 5ms of latency to records sent in the absence of a load.