Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Kafka logs

Highlighted

Kafka logs

Kafka is logging under /var/log.kafka but space it is taking is huge . We tried to suppress INFO and WARN messages from logs but still I could see WARN messages in the log files .

Please find below kafka logs config we have in Advanced kafka-log4j

log4j.logger.kafka=ERROR, kafkaAppender

log4j.logger.kafka.network.RequestChannel$=WARN,

requestAppender log4j.additivity.kafka.network.RequestChannel$=false

Do I need to comment log4j.logger.kafka.network.RequestChannel$=WARN, in order to suppress warning as well .

Note : I wanted only Error messages in the log files .

8 REPLIES 8

Re: Kafka logs

Super Mentor

@Anurag Mishra

I see a very similar thread of yours here: https://community.hortonworks.com/questions/170028/supress-info-and-warn-from-logs.html

Didn't the following setting work for you? Are you setill seeing WARN / INFO messages?

log4j.logger.kafka=ERROR, kafkaAppender

.

If you want to filter/Suppress any specific entry from the log then please share the exact WARNING message that you want to suppress.

Re: Kafka logs

Super Mentor

@Anurag Mishra

If you really want to suppress the logs containing any specific string which is getting repeated again and again then you might want to use the Log4j "StringMatchFilter" as described in the following article:

https://community.hortonworks.com/articles/131369/how-to-suppress-some-specific-unwanted-log-entries...

Re: Kafka logs

@Jay Kumar SenSharma

Yes Jay, after making changes as you mentioned in different thread , Still I am seeing WARN messages in the logs , and log size is growing with rapid speed .

Re: Kafka logs

@Jay Kumar SenSharma

is log4j.logger.kafka.network.RequestChannel$=WARN, property enabling logging WARN messages ?

Re: Kafka logs

Super Mentor

@Anurag Mishra

This line means for the package ending with "kafka.network.RequestChannel" Will long WARNIN or above logging level means WARNING , ERROR, FATAL, CRITICAL ,,etc

But it will not log INFO

Re: Kafka logs

I am still not able to understand why I am seeing WARN messages spite of setting log4j.logger.kafka=ERROR, kafkaAppender

Re: Kafka logs

@Jay Kumar SenSharma

and what If I change log4j.logger.kafka.network.RequestChannel$ to ERROR ?

Re: Kafka logs

@Jay Kumar SenSharma

Hi Jay please find below configs :

log4j.logger.kafka=ERROR, kafkaAppender

log4j.logger.kafka.network.RequestChannel$=WARN, requestAppender

log4j.additivity.kafka.network.RequestChannel$=false

#log4j.logger.kafka.network.Processor=TRACE, requestAppender

#log4j.logger.kafka.server.KafkaApis=TRACE, requestAppender

#log4j.additivity.kafka.server.KafkaApis=false

log4j.logger.kafka.request.logger=WARN, requestAppender

log4j.additivity.kafka.request.logger=false

log4j.logger.kafka.controller=TRACE, controllerAppender

log4j.additivity.kafka.controller=false

log4j.logger.kafka.log.LogCleaner=ERROR, cleanerAppender

log4j.additivity.kafka.log.LogCleaner=false

log4j.logger.state.change.logger=TRACE, stateChangeAppender

log4j.additivity.state.change.logger=false