Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Where are log files in kafka located? Where are zookeeper logs located?

Highlighted

Where are log files in kafka located? Where are zookeeper logs located?

New Contributor

[The following question was moved here because the thread it was originally posted to 12-11-2019 was marked Solved  01-26-2018 —Moderator]

 

I don't have /var/log/kafka in my setup. Where else might I find the actual kafka logs? Same question for zookeeper

 

4 REPLIES 4
Highlighted

Re: Where are log files in kafka located? Where are zookeeper logs located?

Super Mentor

@TJSully 

On the Kafka Host where Kafka Broker is running can you please try running the following command to see the value of "kafka.logs.dir" property.

# ps -ef | grep kafka

.

Example Output in my case:

kafka    21831     1 41 22:58 ?        00:00:03 /usr/jdk64/jdk1.8.0_112/bin/java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Xloggc:/var/log/kafka/kafkaServer-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/var/log/kafka -Dlog4j.configuration=file:/usr/hdp/2.6.5.0-292kafka/bin/../config/log4j.properties .................. kafka.Kafka /usr/hdp/2.6.5.0-292/kafka/config/server.properties

 

Notice the value of property "kafka.logs.dir" it might be a different value in your case.

.

 

.

Highlighted

Re: Where are log files in kafka located? Where are zookeeper logs located?

Super Mentor

@TJSully 

Similarly for the Zookeeper process you can search for "zookeeper.log.dir" property in the output:

# ps -ef | grep zookeeper | grep zookeeper.log.dir --color
Highlighted

Re: Where are log files in kafka located? Where are zookeeper logs located?

New Contributor

Didn't find anything for zookeeper. Results were:

 

$ ps -ef | grep zookeeper | grep zookeeper.log.dir --color
cvrt_apv 16630 14790 0 13:50 pts/1 00:00:00 grep --color=auto zookeeper.log.dir --color
/home/cvrt_apv:
$

Highlighted

Re: Where are log files in kafka located? Where are zookeeper logs located?

New Contributor

The only result I found with "logs" in ti was the following. I went as far as the bin directory and had to stop as there were no more directories to follow.

Dkafka.logs.dir=/opt/kafka/app/bin/../logs

 

Also found the following, used $ps -ef | grep logs, but same as above, no additional directories in the bin directory

Xloggc:/opt/kafka/app/bin/../logs/kafkaServer-gc.log

 

By searching around I found the following directory /opt/kafka/app/logs but the files there don't look like activity logs. I'm trying to find why my brokers quit consuming the incoming feeds. I see files like server.log.2019-12-12-14

and state-change.log.2019-12-05-18

Don't have an account?
Coming from Hortonworks? Activate your account here