Hi, Can you help me understand what is the role of syslog-ng in your configuration? Is syslog-ng being used to feed syslog messages to the ListenSyslog processor? syslog-ng can also write directly to HDFS (https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/configuring-destinations-hdfs.html) and here you can find details about its performance and tips for tuning its performance: https://www.balabit.com/blog/testing-performace-log-streaming-hdfs-syslog-ng/ , Hi, You refer in the question to syslog-ng, can you clarify what is the role of syslog-ng in your configuration? Is syslog-ng being used to feed syslog messages to the ListenSyslog processor? syslog-ng can also directly write messages to HDFS (https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/configuring-destinations-hdfs.html) and you can find performance measurement results and optimisation tips here: https://www.balabit.com/blog/testing-performace-log-streaming-hdfs-syslog-ng/
... View more
