Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS replication failing in BDR

HDFS replication failing in BDR

Explorer

The HDFS replication is failing even after the successful completion of the MapReduce job with the following error:

 

ERROR distcp.DistCp: Job failed to copy 709 files/dirs. Please check Copy Status.csv file or Error Status.csv file for error messages

INFO distcp.DistCp: Used diff: false

INFO distcp.DistCp: Distcp failed to copy some files/dirs, skipping delete and rename snapshot operations on Source

INFO distcp.DistCp: Distcp failed to copy some files/dirs, skipping delete and rename snapshot operations on Target

 

The cluster is running with CDH 5.15.1. There are enough resources available in the destination cluster for the running of MapReduce job for DistCp. This is critical. Please assist.

5 REPLIES 5

Re: HDFS replication failing in BDR

New Contributor

Hi,

 

QQs:

 

Was it working earlier?

Is this on enterprise CM?

Did you try through CLI?

 

Thanks

Re: HDFS replication failing in BDR

Explorer
1. Nope. Setting it up for the first time. Both the clusters are SSL enabled using self-signed certs. Peering has been done successfully.
2. Yes
3. Nope. Using the replication schedules under the Backup tab.
Highlighted

Re: HDFS replication failing in BDR

Community Manager

Hi @urbanlad20 ,

 

Could you please share the Copy Status.csv file and/or Error Status.csv file so we can find out more details of what happened?

 

Thanks,

Li

Li Wang, Technical Resolution Manager


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community:

Terms of Service

Community Guidelines

How to use the forum

Re: HDFS replication failing in BDR

Explorer

There is nothing shown in the Error status file and its kind of a dead end. Also, KMS has been enabled in the Source cluster. Tried manually doing a DistCp -update which was not working initially, but after removing the HDFS user from the blacklist in the kms-acls.xml file, it worked out fine.
Attached is the snap of the Error-status.csv fileimage.png

Re: HDFS replication failing in BDR

New Contributor

We have similar issue with the following

 

19/10/17 14:33:18 ERROR distcp.DistCp: Job failed to copy 9 files/dirs. Please check Copy Status.csv file or Error Status.csv file for error messages
19/10/17 14:33:18 INFO distcp.DistCp: Used diff: false

 

Where the Status.csv file is located?

Don't have an account?
Coming from Hortonworks? Activate your account here