02-13-2017 08:55 PM
02-14-2017 02:18 AM
Looking at one of the running jobs conf and see the following with replication factor 3:
02-17-2017 11:40 PM - edited 02-18-2017 09:23 AM
Any other ideas?
The more intersting in the issue that it's happens only for the output of specific jobs and notf or all the HDFS.
Is there any way to set that the new written files to specific dir to be with specific replication factor?
02-23-2017 12:49 PM
Digging down in the cluster, i found one of the application that runs outside of the hadoop cluster has clients that make hdfs dfs -put to the hadoop cluster, these clients weren't have hdfs-site.xml and it got the default replication factor for the cluster, what i did? tested the hdfs dfs -put from a cleint server in my cluster and the client out side the cluster and notice the client outside the cluster put files with replication factor 3, to solve the issue i added hdfs-site.xml to each of the clients outside the cluster and override the default replication factor at the file.