- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
NameNode alerting on Blocks under replicated event dfs replication changed at NN and DNs.
- Labels:
-
HDFS
Created 01-30-2017 02:27 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
When i Run fsck on my cluster i got that several blocks under replicated and the target replication is 3 even i changed the dfs.replication to NN and DNs to 2.
My cluster status
Live Nodes | : | 3 (Decommissioned: 1) |
Total size: 1873902607439 B
Total dirs: 122633
Total files: 117412
Total blocks (validated): 119731 (avg. block size 15650939 B)
Minimally replicated blocks: 119731 (100.0 %)
Over-replicated blocks: 68713 (57.38948 %)
Under-replicated blocks: 27 (0.022550551 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 2
Average block replication: 2.5738947
Corrupt blocks: 0
Missing replicas: 27 (0.011274004 %)
Number of data-nodes: 3
Number of racks: 1
FSCK ended at Mon Jan 30 04:59:23 EST 2017 in 2468 milliseconds
NN and DNs hdfs.site.xml:
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
The only change i did that i deco one of the servers and it's now in decomissioned state, even i set replication factor for all HDFS manually to 2 but still see the new written blocks are alerted on target replica as 3, also i ensure that the mapred submit replica also 2 in JT:
<property>
<name>mapred.submit.replication</name>
<value>2</value>
</property>
Any insights?
Created 02-13-2017 08:55 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
<property>
<name>dfs.replication</name>
<value>2</value>
<final>true</final>
</property>
Created 01-30-2017 06:26 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Anyone who can help here
Created 01-31-2017 01:14 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So if you have under-replicated blocks asking for 3 replica then you need to alter these blocks directly.
Check the command line "hdfs dfs -setrep"
Created 01-31-2017 11:23 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 02-03-2017 01:12 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Also post the hdfs-site.xml if possible from the client you are one or were you upload from?
It just dawned on me that you stated that the NN and DN have replication set to 2 but if the clients uploading the data still have it set to 3 in their hdfs-site.xml then the file(s) will have a factor of 3.
Created 02-03-2017 08:10 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
replication definition.
In which cases NN decides the replication and on which client is deciding,
I will check all clients which are working with my cluster and check the
hdfs-site.xml.
Created 02-03-2017 09:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In short, the client always defines the replication factor.
As an example, the Terasort application provide with the distribution has the replication factor set to 1. Unless you change it the terasort output is written to HDFS with only one replica.
The Namenode just keeps track of how many replicas there should be and are, etc.
Created 02-03-2017 09:41 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
files and folders and running -ls command will be difficult to follow and
got the block replicas.
Created 02-03-2017 09:45 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
This just prints out the factor and not the file name.
Created 02-03-2017 09:53 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have /data and /code dirs and under /data i have data1 and data2 dir and
under these data1 and 2 dirs i have files, can you please provide the
command that i ca use?
