Standby Name node down????

[#1]

my standby namenode was down. when i try to start the standby namenode from ambari it was come up. what would be the reason for standby namenode down. please let me know where can i find standby namenode logs.

Please help me..

[#2]

@kanna k

Can you check the /var/log/message is there a useful message? The namenode log should be on the standby NameNode !

[#3]

i didn't find any word called namenode in /var/logmessage

[#4]

@kanna k

We are not looking for the NameNode logs here but I suspect some memory leak that brings down the Standby namenode look for something memory related like

kernel: Memory cgroup out of memory: 
kernel: Killed process xxxx

HTH

[#5]

sry @Geoffrey Shelton Okot i didn't find the words like memory ,cgroup ,kernel,killed.and at present there is no alerts on Ambari UI how to find the root cause....

[#6]

@kanna k

Have you checked on the standby namenode in /var/log/hadoop/hdfs

[#7]

@Geoffrey Shelton Okot

i have checked but i didn't find any logs in that directory

[#8]

Standby is again went down today can you please help me solve this

[#9]

@kanna k

Can you find the Standby NN logs on the standby node please share the log & out file. IS NTP running in your cluster can you check that all the cluster time is in sync?

[#10]

@kanna k, you can find out location of standby namenode logs from hadoop-env.sh. Look for HADOOP_LOG_DIR value to find out correct location of the log .

Example:

export HADOOP_LOG_DIR=/var/log/hadoop/$USER

In this example, standby namenode log will be present at /var/log/hadoop/hdfs dir.

[#11]

@kanna k

Any updates?