metron - kafka-log/pcap0 consume all space

[#1]

hi,

I followed in the instruction from metron wiki installation 4.0 to setup a 3 node cluster. Everything is working fine, however every other day I get one of the node 100% full b/c of the /kafka-log/pcap-0.

Why is this the case? is it because kafka is not processing the messages coming in for pcap topic fast enough?

How can I eliminate this issue in the future? I'm thinking I need to reconfigure or change something in kafka to be more efficient?

Any insight/idea is greatly appreciated.

[#2]

Are you ingesting pcap data (looks like you are)? This is a high volume data source so you likely need to scale Kafka by adding more nodes/partitions. Are the other nodes filling up too? How many Kafka brokers do you have? A 3 node cluster is really small (especially for something like ingesting pcap) so you're likely undersized anyways.

[#3]

THank you @rmerriman for your response.

I have 3 Kafka brokers, one on each node. It looks like only one node contains the data for topic pcap and only that particular node is filled up. I modified kafka configuration so that we have three partition for each topic instead of the default of 1. I also modified the retention time and byte size to be small.

How many node roughly would you recommend for a cluster to ingest pcap data?

thanks again for all feedback