Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)
Guru

This article has been set on a HDP 2.5.3 version, you may consider adjusting some parameters to reflect your actual version.

We'll here set Kafka loglevel through the Logging MBean with jConsole.

For that, the first step is to enable JMX access:

  1. add in Kafka configs/kafka-env template
    export KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.local.only=false"
    export JMX_PORT="9999"
    
  2. For avoiding JMX port conflicts like mentioned in https://community.hortonworks.com/articles/73750/kafka-jmx-tool-is-failing-with-port-already-in-use...., let’s modify /usr/hdp/current/kafka/bin/kafka-run-class.sh on all broker nodes: replace
# JMX port to use
if [ $JMX_PORT ]; then
  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT"
fi

with

# JMX port to use
if [ $ISKAFKASERVER = "true" ]; then
  JMX_REMOTE_PORT=$JMX_PORT
else
  JMX_REMOTE_PORT=$CLIENT_JMX_PORT
fi
if [ $JMX_REMOTE_PORT ]; then
  KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_REMOTE_PORT"
fi

After brokers has been restarted, lets modify the logLevel with jConsole:

$ jconsole <BROKER_FQDN>:<JMX_PORT>

It launches a jconsole window, asking for Retry insecurely, go ahead with that

go to the Mbeans tab then Kafka/kafka.log4jController/Attributes, and double-click on the Value of Loggers to get all Log4j controllers

93776-screen-shot-2018-12-03-at-170013.png

You can see the kafka logger above those presented is set to INFO.

We can check it using the getLogLevel Operations entering the kafka loggerName

93777-screen-shot-2018-12-03-at-170750.png

Fortunately, you can also set the value without restarting with the setLogLevel operation, putting in DEBUG or TRACE for example.

93778-screen-shot-2018-12-03-at-170917.png

850 Views
0 Kudos
Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
2 of 2
Last update:
‎08-17-2019 05:35 AM
Updated by:
 
Contributors
Top Kudoed Authors