How to change Kafka log level on the fly

[#1]

This article has been set on a HDP 2.5.3 version, you may consider adjusting some parameters to reflect your actual version.

We'll here set Kafka loglevel through the Logging MBean with jConsole.

For that, the first step is to enable JMX access:

1. add in Kafka configs/kafka-env template
   

   export KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false"
   export JMX_PORT="9999"
   

2. For avoiding JMX port conflicts like mentioned in https://community.hortonworks.com/articles/73750/kafka-jmx-tool-is-failing-with-port-already-in-use...., let’s modify /usr/hdp/current/kafka/bin/kafka-run-class.sh on all broker nodes: replace

# JMX port to use
if [ $JMX_PORT ]; then
  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"
fi

with

# JMX port to use
if [ $ISKAFKASERVER = "true" ]; then
  JMX_REMOTE_PORT=$JMX_PORT
else
  JMX_REMOTE_PORT=$CLIENT_JMX_PORT
fi
if [ $JMX_REMOTE_PORT ]; then
  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_REMOTE_PORT"
fi

After brokers has been restarted, lets modify the logLevel with jConsole:

$ jconsole <BROKER_FQDN>:<JMX_PORT>

It launches a jconsole window, asking for Retry insecurely, go ahead with that

go to the Mbeans tab then Kafka/kafka.log4jController/Attributes, and double-click on the Value of Loggers to get all Log4j controllers

You can see the kafka logger above those presented is set to INFO.

We can check it using the getLogLevel Operations entering the kafka loggerName

Fortunately, you can also set the value without restarting with the setLogLevel operation, putting in DEBUG or TRACE for example.