- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Cluster regain Help: One of my Namenode directory lost (means two directories are kept for name node back up in that one node has lost).
- Labels:
-
Apache Ambari
-
Apache Hadoop
Created on ‎05-13-2016 04:47 PM - edited ‎09-16-2022 03:19 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Second,Ambari was not able to start because of /var/log got filled. Third Connection exception is giving when we try to type hadoop command. we want to regain back the cluster any suggestions to regain steps manually since Ambari is not working
Created ‎05-13-2016 05:21 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1. NN data lost.
Is it that the disk with NN directories crashed or have you deleted them? Is this with HA or non-HA. With non-HA, if both data directories of NN have no data, then you will run into data loss issues. You can revive and get to some state from secondary NN data directories but can not guarantee no data loss. If there is no useful data, you can always issue a NN format and start fresh. (You will need to manually update tez and mapreduce apps, you can get the information from manual install documentation)
2. Ambari not starting.
Clean up /var/log and start it back.
3. Most likely HDFS services are not up. Filled up disk can kill processes. Once ambari is up, see which one is running and which one is not. Againt if NN data is lost, then NN will not start up.
Created ‎05-13-2016 05:21 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1. NN data lost.
Is it that the disk with NN directories crashed or have you deleted them? Is this with HA or non-HA. With non-HA, if both data directories of NN have no data, then you will run into data loss issues. You can revive and get to some state from secondary NN data directories but can not guarantee no data loss. If there is no useful data, you can always issue a NN format and start fresh. (You will need to manually update tez and mapreduce apps, you can get the information from manual install documentation)
2. Ambari not starting.
Clean up /var/log and start it back.
3. Most likely HDFS services are not up. Filled up disk can kill processes. Once ambari is up, see which one is running and which one is not. Againt if NN data is lost, then NN will not start up.
