Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Where are log files in kafka located? Where are zookeeper logs located?

avatar
New Contributor

[The following question was moved here because the thread it was originally posted to 12-11-2019 was marked Solved  01-26-2018 —Moderator]

 

I don't have /var/log/kafka in my setup. Where else might I find the actual kafka logs? Same question for zookeeper

 

4 REPLIES 4

avatar
Master Mentor

@TJSully 

On the Kafka Host where Kafka Broker is running can you please try running the following command to see the value of "kafka.logs.dir" property.

# ps -ef | grep kafka

.

Example Output in my case:

kafka    21831     1 41 22:58 ?        00:00:03 /usr/jdk64/jdk1.8.0_112/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xloggc:/var/log/kafka/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/var/log/kafka -Dlog4j.configuration=file:/usr/hdp/2.6.5.0-292kafka/bin/../config/log4j.properties .................. kafka.Kafka /usr/hdp/2.6.5.0-292/kafka/config/server.properties

 

Notice the value of property "kafka.logs.dir" it might be a different value in your case.

.

 

.

avatar
Master Mentor

@TJSully 

Similarly for the Zookeeper process you can search for "zookeeper.log.dir" property in the output:

# ps -ef | grep zookeeper | grep zookeeper.log.dir --color

avatar
New Contributor

Didn't find anything for zookeeper. Results were:

 

$ ps -ef | grep zookeeper | grep zookeeper.log.dir --color
cvrt_apv 16630 14790 0 13:50 pts/1 00:00:00 grep --color=auto zookeeper.log.dir --color
/home/cvrt_apv:
$

avatar
New Contributor

The only result I found with "logs" in ti was the following. I went as far as the bin directory and had to stop as there were no more directories to follow.

Dkafka.logs.dir=/opt/kafka/app/bin/../logs

 

Also found the following, used $ps -ef | grep logs, but same as above, no additional directories in the bin directory

Xloggc:/opt/kafka/app/bin/../logs/kafkaServer-gc.log

 

By searching around I found the following directory /opt/kafka/app/logs but the files there don't look like activity logs. I'm trying to find why my brokers quit consuming the incoming feeds. I see files like server.log.2019-12-12-14

and state-change.log.2019-12-05-18