Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDFS replication failing in BDR

avatar
Explorer

The HDFS replication is failing even after the successful completion of the MapReduce job with the following error:

 

ERROR distcp.DistCp: Job failed to copy 709 files/dirs. Please check Copy Status.csv file or Error Status.csv file for error messages

INFO distcp.DistCp: Used diff: false

INFO distcp.DistCp: Distcp failed to copy some files/dirs, skipping delete and rename snapshot operations on Source

INFO distcp.DistCp: Distcp failed to copy some files/dirs, skipping delete and rename snapshot operations on Target

 

The cluster is running with CDH 5.15.1. There are enough resources available in the destination cluster for the running of MapReduce job for DistCp. This is critical. Please assist.

7 REPLIES 7

avatar
Explorer

Hi,

 

QQs:

 

Was it working earlier?

Is this on enterprise CM?

Did you try through CLI?

 

Thanks

avatar
Explorer
1. Nope. Setting it up for the first time. Both the clusters are SSL enabled using self-signed certs. Peering has been done successfully.
2. Yes
3. Nope. Using the replication schedules under the Backup tab.

avatar
Guru

Hi @urbanlad20 ,

 

Could you please share the Copy Status.csv file and/or Error Status.csv file so we can find out more details of what happened?

 

Thanks,

Li

Li Wang, Technical Solution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

avatar
Explorer

There is nothing shown in the Error status file and its kind of a dead end. Also, KMS has been enabled in the Source cluster. Tried manually doing a DistCp -update which was not working initially, but after removing the HDFS user from the blacklist in the kms-acls.xml file, it worked out fine.
Attached is the snap of the Error-status.csv fileimage.png

avatar
New Contributor

Can you please let us know where is the location for Status.csv file.
I searched in HDFS location and Linux OS file system but didn't get.
FYI: I searched in exact data node machine where this distcp operation failed. 
Any hint would be hepful for my project. 

During DR job, am getting this message <Hive Table Data Replication
"ERROR distcp.DistCp: Job failed to copy files. Please check Copy Status.csv"

avatar
Community Manager

@Bulu As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
New Contributor

We have similar issue with the following

 

19/10/17 14:33:18 ERROR distcp.DistCp: Job failed to copy 9 files/dirs. Please check Copy Status.csv file or Error Status.csv file for error messages
19/10/17 14:33:18 INFO distcp.DistCp: Used diff: false

 

Where the Status.csv file is located?