Zookeeper Server keeps stopping after I restart with Ambar

[#1]

I'm running hdp 2.3.6 on AWS.

I restart zookeeper server & it starts without any error but quickly stops. The stderr has no entries.

I'd appreciate any help to debug this issues.

Thanks in advance,

Sai Dileep.

[#2]

It could be your configuration problem if you don't find any logs even after this setting. or you may be configured different log directory location.

Share the output of "/etc/zookeeper/conf/zookeeper-env.sh" ?

Also, you are missing the log patterns. Please append to log4j.properties as highlighted here.

Ambari -> Zookeeper -> Configs -> Advanced zookeeper-log4j,

..

log4j.appender.ROLLINGFILE.MaxBackupIndex=10

log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n

[#3]

Output of "zookeeper-env.sh" :

export JAVA_HOME=/usr/jdk64/jdk1.8.0_112
export ZOOKEEPER_HOME=/usr/hdp/current/zookeeper-server
export ZOO_LOG_DIR=/var/log/zookeeper
export ZOOPIDFILE=/var/run/zookeeper/zookeeper_server.pid
export SERVER_JVMFLAGS=-Xmx1024m
export JAVA=$JAVA_HOME/bin/java
export CLASSPATH=$CLASSPATH:/usr/share/zookeeper/*

[#4]

Ok, ensure you have right permission & ownership of the directory.

/var/log/zookeeper

#ls -lrth /var/log/ | grep zookeeper

drwxr-xr-x. 3 zookeeper hadoop 4.0K Nov 23 13:32 zookeeper

Did you check? after the pattern update.

Ambari -> Zookeeper -> Configs -> Advanced zookeeper-log4j,

..

log4j.appender.ROLLINGFILE.MaxBackupIndex=10

log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n

[#5]

Output of

ls -lrth /var/log/ | grep zookeeper

is

drwxr-xr-x 2 zookeeper hadoop 4.0K نوف 21 16:18 zookeeper

Still nothing in /var/log/zookeeper

[#6]

@Maryem Mary

It appears some logs got generated. Your directory size is 4KB now.

#ls -lrth /var/log/zookeeper ??

[#7]

I dont what i have missed but the dir :

/var/log/zookeeper and

/usr/share/zookeeper a

are empty!!

[#8]

@Sandeep Kumar

I'm heaving the same issue with zookeeper being stopped a few seconds after startup. My problem though is that it seems the changes made to log4j properties through Ambari are not getting propagated. Here the "Advanced zookeeper log4j":

# DEFAULT: console appender only #log4j.rootLogger=INFO, CONSOLE, ROLLINGFILE # Example with rolling log file log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE # Example with rolling log file and tracing #log4j.rootLogger=TRACE, CONSOLE, ROLLINGFILE, TRACEFILE # # Log INFO level and above messages to the console # log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender log4j.appender.CONSOLE.Threshold=INFO log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n # # Add ROLLINGFILE to rootLogger to get log file output # Log DEBUG level and above messages to a log file log4j.appender.ROLLINGFILE=org.apache.log4j.RollingFileAppender log4j.appender.ROLLINGFILE.Threshold=DEBUG log4j.appender.ROLLINGFILE.File={{zk_log_dir}}/zookeeper.log # Max log file size of 10MB log4j.appender.ROLLINGFILE.MaxFileSize={{zookeeper_log_max_backup_size}}MB # uncomment the next line to limit number of backup files #log4j.appender.ROLLINGFILE.MaxBackupIndex={{zookeeper_log_number_of_backup_files}} log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L] - %m%n # # Add TRACEFILE to rootLogger to get log file output # Log DEBUG level and above messages to a log file log4j.appender.TRACEFILE=org.apache.log4j.FileAppender log4j.appender.TRACEFILE.Threshold=TRACE log4j.appender.TRACEFILE.File=zookeeper_trace.log log4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout ### Notice we are including log4j's NDC here (%x) log4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} - %-5p [%t:%C{1}@%L][%x] - %m%n

But yet ps -ef | grpe zookeper:

zookeep+ 18598 1 4 23:49 ? 00:00:00 /usr/jdk64/jdk1.8.0_112/bin/java -Dzookeeper.log.dir=/xxx/zk/logs -Dzookeeper.log.file=zookeeper-zookeeper-server-xxxxxxxxxxx.log -Dzookeeper.root.logger=INFO,ROLLINGFILE .....

Note that neither DEBUG, nor new log file name (zookeeper.log) have not been propagated by ambari

Is there any property missing?

Thanks,

Alex