We had a HDF Cluster with customized settings to route all the audit log to file system
Long term plan is to feed the logs to central platform, but currently these logs are sitting at remote site, and the normal kafka message reading audit log is filling up the server disk very quickly.
-rw-r--r-- 1 kafka hadoop 20067988960 Jan 3 16:28 ranger_kafka_audit.log
# see the difference of the size after a few min. (almost 2G in 7 min)
-rw-r--r-- 1 kafka hadoop 18363271828 Jan 3 16:21 ranger_kafka_audit.log
We are looking for a temp solution to only write the audit log with "forbidden" tag to the file.
Does anyone have idea of customize the configuration so that we can control the content to be logged?