Member since
11-19-2016
3
Posts
2
Kudos Received
1
Solution
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 1952 | 11-25-2016 02:54 PM |
11-25-2016
02:54 PM
1 Kudo
[SOLVED] Hello, First I read this awesome explanation of HDFS Metada directories Then I regained access to the ambari-server (which was switched off for some reason) and launched the SecondaryNameNode start script via ambari-agent. As I watched the logfiles I saw it was doing copy of the edits and fsimage from PrimaryNameNode, and the purging old edits. Now the metadata directories structure of my SecondaryNameNode is the same as on PrimaryNameNode and it is doing checkpoint update every 6h. -rw-r--r-- 1 hdfs hadoop 152M 11-24 12:58 edits_0000000000018021905-0000000000019484203
-rw-r--r-- 1 hdfs hadoop 14M 11-24 18:57 edits_0000000000019484204-0000000000019605416
-rw-r--r-- 1 hdfs hadoop 14M 11-25 00:57 edits_0000000000019605417-0000000000019726471
-rw-r--r-- 1 hdfs hadoop 14M 11-25 06:57 edits_0000000000019726472-0000000000019847549
-rw-r--r-- 1 hdfs hadoop 14M 11-25 12:57 edits_0000000000019847550-0000000000019968584
-rw-r--r-- 1 hdfs hadoop 17M 11-25 06:57 fsimage_0000000000019847549
-rw-r--r-- 1 hdfs hadoop 62 11-25 06:57 fsimage_0000000000019847549.md5
-rw-r--r-- 1 hdfs hadoop 17M 11-25 12:57 fsimage_0000000000019968584
-rw-r--r-- 1 hdfs hadoop 62 11-25 12:57 fsimage_0000000000019968584.md5
-rw-r--r-- 1 hdfs hadoop 205 11-25 12:57 VERSION
I the nearest future I will plan to setup a new HDFS cluster with ambari, using latest HDP with HA enabled, and copy the data from the old HDFS. Thank You very much for your help, Regards, JK
... View more
11-22-2016
10:36 PM
Thanx a lot for your answer. Regarding last paragraph: Well,
basically I need SecondaryNameNode to be fully operational because as
far as I can see there is only PrimaryNameNode (plus dataNodes of
course) running on "my inherited infrastructure" and in case of
crash/restart of this server, according to the documentation, it can
take a lot (how much exactly assuming that the tarred size of
/hadoop/hdfs/namenode/current/ is 300MB ?) time to start a
PrimaryNameNode process. The current files in
/haddop/hdfs/namenode/current/ directory gives us information that
SecondaryNameNode owa operational till 19.02.2016. After its
(SecondaryNameNode) crash edit_ files became bigger (expected behavior).
-rw-r--r-- 1 hdfs hadoop 33K 2016-02-18 edits_0000000000004018182-0000000000004018420
-rw-r--r-- 1 hdfs hadoop 33K 2016-02-19 edits_0000000000004018421-0000000000004018656
-rw-r--r-- 1 hdfs hadoop 35K 2016-02-19 edits_0000000000004018657-0000000000004018904
-rw-r--r-- 1 hdfs hadoop 33K 2016-02-19 edits_0000000000004018905-0000000000004019140
-rw-r--r-- 1 hdfs hadoop 33K 2016-02-19 edits_0000000000004019141-0000000000004019376
-rw-r--r-- 1 hdfs hadoop 269M 06-16 11:15 edits_0000000000004019377-0000000000006019392
-rw-r--r-- 1 hdfs hadoop 226M 09-28 08:20 edits_0000000000006019393-0000000000008019731
-rw-r--r-- 1 hdfs hadoop 221M 09-29 22:20 edits_0000000000008019732-0000000000010019773
-rw-r--r-- 1 hdfs hadoop 221M 10-01 12:20 edits_0000000000010019774-0000000000012020578
-rw-r--r-- 1 hdfs hadoop 201M 10-18 23:25 edits_0000000000012020579-0000000000014020981
-rw-r--r-- 1 hdfs hadoop 231M 10-24 09:45 edits_0000000000014020982-0000000000016021135
-rw-r--r-- 1 hdfs hadoop 221M 11-12 05:30 edits_0000000000016021136-0000000000018021904
-rw-r--r-- 1 hdfs hadoop 92M 11-21 16:03 edits_inprogress_0000000000018021905
-rw-r--r-- 1 hdfs hadoop 22M 2016-02-19 fsimage_0000000000004019140
-rw-r--r-- 1 hdfs hadoop 62 2016-02-19 fsimage_0000000000004019140.md5
-rw-r--r-- 1 hdfs hadoop 22M 2016-02-19 fsimage_0000000000004019376
-rw-r--r-- 1 hdfs hadoop 62 2016-02-19 fsimage_0000000000004019376.md5
-rw-r--r-- 1 hdfs hadoop 9 11-12 05:30 seen_txid
-rw-r--r-- 1 hdfs hadoop 205 2014-11-04 VERSION
I would love to implement the StandbyName Node/Nodes if it is
implementable without PrimaryNameNode restart (it is a production HDFS
:) and my management did not have a clue about importance of theirs own
HDFS system 🙂 ). I do not know the reason why it was
stoped/killed, the person responsible for it is unreachable, so I need
to do a small investigation. I see that the system was started via
ambari. So to summarize it all: Is it better in this case (SecondaryNameNode is not running for 10months anyway) to: 1. Start a SecondaryNameNode manually and wait for synchronization of edit file with fsimage with "fingers crossed" 🙂 ? 2. Reconfigure the old SecondaryNameNode and start a StandbyNameNode manually on the same server instead of SecondaryNameNode ? This is a hadoop 2.4.1 so AFAIK there is only one StandbyNameNode posibble, please correct me if I am wrong. Thank you, Jan K
... View more
11-19-2016
02:44 AM
1 Kudo
Hello, At my new work I inherited a Hadoop (2.4.1) cluster with dead Secondary NameNode since February. AFAIK I need to resync image by launching SecondaryNameNode manually: hadoop-daemon.sh start secondarynamenode But I am wondering if such a long gap will impact image resync process? Will PrimaryNameNode be responsive? Regards, JK ,
... View more
Labels:
- Labels:
-
Apache Hadoop