Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Namenode and Journalnodes are not restarting

Solved Go to solution
Highlighted

Namenode and Journalnodes are not restarting

Explorer

Hello 

 

Can some one please help me on this issue,

My two journal nodes are not in sync and i cant able to restart them and both the namenodes are down.

I can see below logs in NN,

 

FATAL org.apache.hadoop.hdfs.server.namenode.FSEditLog
Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [10.10.10.10:8485, 10.10.10.11:8485, 10.10.10.09:8485], stream=null))
org.apache.hadoop.hdfs.qjournal.client.QuorumException: Got too many exceptions to achieve quorum size 2/3. 1 successful responses:
10.10.10.09:8485: lastPromisedEpoch: 69
httpPort: 8480 fromURL: "http://Master_Host(10.10.10.09):8480"

 

 

When i use to do restart of JN and not happening,

And below are the Journalnode logs,

 

ERROR org.apache.hadoop.hdfs.qjournal.server.JournalNode
RECEIVED SIGNAL 15: SIGTERM
WARN org.apache.hadoop.metrics2.impl.MetricsConfig
Cannot locate configuration: tried hadoop-metrics2-journalnode.properties,hadoop-metrics2.properties

 

 

Can some one please suggest me.

 

Thanks,

Vinod

 

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Namenode and Journalnodes are not restarting

Explorer

Hello thanks for your responce, Actually the issue is resolved and its JournalNodeEdits directory permission issue.

So i have modified it and restarted Jornalnodes and started successfully.

NN's are also came back and able to see all other dependent services.

 

Thanks again :)

 

2 REPLIES 2

Re: Namenode and Journalnodes are not restarting

Cloudera Employee

Hello,

 

Error: recoverUnfinalizedSegments failed for required journal (JournalAndStream(mgr=QJM to [10.10.10.10:8485, 10.10.10.11:8485, 10.10.10.09:8485], stream=null))

 

As you can see above that instead of FQDN, the IP shows up, which means that when the JN is trying to talk to DNS to resolve the IP to the FQDN, it failed. It looks like your DNS have issues resolving the IPs. Please look into it.

Re: Namenode and Journalnodes are not restarting

Explorer

Hello thanks for your responce, Actually the issue is resolved and its JournalNodeEdits directory permission issue.

So i have modified it and restarted Jornalnodes and started successfully.

NN's are also came back and able to see all other dependent services.

 

Thanks again :)

 

Don't have an account?
Coming from Hortonworks? Activate your account here