@Hoang Le
1. I know default of replication block is 3. But when I configure dfs.replication=1, Do it affected to cluster performance.
Since you are not replicating, your writes will be faster at the expense of significant risk of data loss as well as read performance. Your reads can be slow because your data might happen to be on a node experiencing issues with no other block available as well as job failure in case of just one node failure.
2. I have a lot of data with configure dfs.replication=1, and now I change configure to dfs.replication= 3. So my data will auto replicate or I have to build my data again to replication running. I need to be sure because my data is very important.
Use setrep to change replication factor for existing files. It will replicate existing data (you will have to provide the path).
hadoop fs -setrep [-R] [-w] <numReplicas> <path>
- hadoop fs -setrep -w 3 /user/hadoop/dir1
- The -R flag is accepted for backwards compatibility. It has no effect.
- The -w flag requests that the command wait for the replication to complete. This can potentially take a very long time.
- Returns 0 on success and -1 on error.
P/S: any best practice for dfs.replication configure.
Always use default replication factor of 3. It provides data resiliency as well as redundancy in case of node failures. It also helps read performance. In rare cases, you can increase replication factor to help even more data distribution to make reads faster.
