How far back do you have files in the dataDirs? There have been some instances where flume holds on to a specific event, that has been delivered, and the checkpoint still references it for some reason. If that is the case, you can stop flume, delete the files in the checkpoints directory, and force a replay of the events in the channel (note this may take a long time, depending on the size of all the logs in the dataDirs). You can set use-fast-replay=true to make it replay faster, but you'll need to increase heap size as well if you choose fast replay. -pd ... View more

The Flume file channel will remove old data log files after they have been delivered by the sink and are no longer needed (it always keeps at least two files, but all others would be eligible for deletion). What is the size of your channel? Is it increasing? Are your sinks able to keep up? You can't purge old data (because flume thinks it still needs it if it hasn't already deleted it), but you can control how much free space is on your disk with the minimumRequiredSpace channel property. You can also use the capacity property to restrict how many events can be in the channel. If your files in the dataDirs are not getting deleted, it's because the sinks haven't delivered events from those files. -pd ... View more

If you are using 0.0.0.0 for listeners, you'll also need an advertised.listeners safety valve to specify what host:port the clients should connect to, as they can't connect to 0.0.0.0. Generally, this is done with external/internal DNS. -pd ... View more

What is your consumer.config file properties, does it have SASL_SSL as the protocol? Can you verify if its listening with SSL correctly: openssl s_client -connect svd0hdatn01:9093 -pd ... View more

Connection refused seems to indicate that this host is not listening on port 9092: svd0hdatn01.<DOMAIN>:9092 You can see what ports kafka is listening on: ps -ef |grep kafka netstat -nap |grep <kafka pid> -pd ... View more

You need the DEBUG level set for the clients, not the brokers. This would be in the "Gateway Logging Threshold", or on the system where you are running the console commands by editing /etc/kafka/conf/tools-log4j.properties -pd ... View more

Try turning on DEBUG for the client commands. If you have a kafka gateway installed on that node, you can set the DEBUG level in the kafka service, otherwise, modify the /etc/kafka/conf/tools-log4j.properties to set the log level to DEBUG, and then run your producer or consumer. Some "retryable" errors when security is enabled keep the clients from properly connecting. -pd ... View more