03-25-2015 03:55 PM
Our Cluster has HA configuration using Quorum-based Storage for HDFS. We have to take one of the Journal node out of the cluster and reprovision it. After we put it back in, we got the issues the Active Namenode could not talk to that journal node, but stil be able to talk with other two journal nodes.
I think about disable HA and then reenable HA again, but I don't know if that is a good idea.
Is any one know how to fix this issue?
03-25-2015 09:12 PM
I was able to fix the issues by creating missing directory on the Journal host where namenode will write the its' edit files. Make sure the VERSION file is created, otherwise you will get org.apache.hadoop.hdfs.qjournal.protocol.JournalNotFormattedException.
12-15-2017 08:19 AM - edited 12-15-2017 08:19 AM
I guess it's a bit late, but in case you haven't managed to resolve this, the property that holds this information is called "dfs.journalnode.edits.dir", and it will point to a directory holding a folder named after a logical name of your cluster. So, for example, if the dfs.journalnode.edits.dir is set to /data/jn, the whole path will be /data/jn/xyz-hdp03.
That folder should contain a folder "current", and a file called "in_use.lock", if it is in use. You can also tell by the age of the lock file, and by searching for the journalnode process in your process list. In case you are using Cloudera Manager, you can find the mentioned property in cluster -> HDFS -> Configuration.
Hope this helps,