Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

The active NameNode is out of sync with this JournalNode issue.

Solved Go to solution
Highlighted

The active NameNode is out of sync with this JournalNode issue.

Contributor

Hello,

Our Cluster has HA configuration using Quorum-based Storage for HDFS. We have to take one of the Journal node out of the cluster and reprovision it. After we put it back in, we got the issues the Active Namenode could not talk to that journal node, but stil be able to talk with other two journal nodes.

I think about disable HA and then reenable HA again, but I don't know if that is a good idea.

Is any one know how to fix this issue?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: The active NameNode is out of sync with this JournalNode issue.

Contributor

I was able to fix the issues by creating missing directory on the Journal host where namenode will write the its' edit files. Make sure the VERSION file is created, otherwise you will get org.apache.hadoop.hdfs.qjournal.protocol.JournalNotFormattedException. 

View solution in original post

4 REPLIES 4
Highlighted

Re: The active NameNode is out of sync with this JournalNode issue.

Contributor

I was able to fix the issues by creating missing directory on the Journal host where namenode will write the its' edit files. Make sure the VERSION file is created, otherwise you will get org.apache.hadoop.hdfs.qjournal.protocol.JournalNotFormattedException. 

View solution in original post

Highlighted

Re: The active NameNode is out of sync with this JournalNode issue.

Explorer

Hi 

 

can you please specify the dir name on the jornal node?

I am getting no sync with name node and don't know what is missing...

Highlighted

Re: The active NameNode is out of sync with this JournalNode issue.

Contributor

Hi reutsc,

 

I guess it's a bit late, but in case you haven't managed to resolve this, the property that holds this information is called "dfs.journalnode.edits.dir", and it will point to a directory holding a folder named after a logical name of your cluster. So, for example, if the dfs.journalnode.edits.dir is set to /data/jn, the whole path will be /data/jn/xyz-hdp03.

That folder should contain a folder "current", and a file called "in_use.lock", if it is in use. You can also tell by the age of the lock file, and by searching for the journalnode process in your process list. In case you are using Cloudera Manager, you can find the mentioned property in cluster -> HDFS -> Configuration.

Hope this helps,

camypaj

Re: The active NameNode is out of sync with this JournalNode issue.

New Contributor

after adding the journal node service stop the services

verify for the presence or Please create the folder till the value it configured till the path mentioned under the config -dfs.journalnode.edits.dir

run the scp -r command from other journal node for Clusternamenode folder

 

Ex: scp -r journal2:/data/hadoop/hdfs/journal/clustername

chown hdfs:hadoop /data/hadoop/hdfs/journal/clustername

Start the Journal node

Observe the Sync of transaction range activity in jounral log and after that edits will be patched as usual 

Don't have an account?
Coming from Hortonworks? Activate your account here