HDFS replication failing in BDR

[#1]

The HDFS replication is failing even after the successful completion of the MapReduce job with the following error:

ERROR distcp.DistCp: Job failed to copy 709 files/dirs. Please check Copy Status.csv file or Error Status.csv file for error messages

INFO distcp.DistCp: Used diff: false

INFO distcp.DistCp: Distcp failed to copy some files/dirs, skipping delete and rename snapshot operations on Source

INFO distcp.DistCp: Distcp failed to copy some files/dirs, skipping delete and rename snapshot operations on Target

The cluster is running with CDH 5.15.1. There are enough resources available in the destination cluster for the running of MapReduce job for DistCp. This is critical. Please assist.

[#2]

Hi,

QQs:

Was it working earlier?

Is this on enterprise CM?

Did you try through CLI?

Thanks

[#3]

1. Nope. Setting it up for the first time. Both the clusters are SSL enabled using self-signed certs. Peering has been done successfully.
2. Yes
3. Nope. Using the replication schedules under the Backup tab.

[#4]

Hi @urbanlad20 ,

Could you please share the Copy Status.csv file and/or Error Status.csv file so we can find out more details of what happened?

Thanks,

Li

[#5]

There is nothing shown in the Error status file and its kind of a dead end. Also, KMS has been enabled in the Source cluster. Tried manually doing a DistCp -update which was not working initially, but after removing the HDFS user from the blacklist in the kms-acls.xml file, it worked out fine.
Attached is the snap of the Error-status.csv file

[#6]

Can you please let us know where is the location for Status.csv file.
I searched in HDFS location and Linux OS file system but didn't get.
FYI: I searched in exact data node machine where this distcp operation failed. 
Any hint would be hepful for my project. 

During DR job, am getting this message <Hive Table Data Replication
"ERROR distcp.DistCp: Job failed to copy files. Please check Copy Status.csv"

[#7]

@Bulu As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.

[#8]

We have similar issue with the following

19/10/17 14:33:18 ERROR distcp.DistCp: Job failed to copy 9 files/dirs. Please check Copy Status.csv file or Error Status.csv file for error messages
19/10/17 14:33:18 INFO distcp.DistCp: Used diff: false

Where the Status.csv file is located?