Reply
Explorer
Posts: 21
Registered: ‎12-13-2016
Accepted Solution

BDR jobs don't replicate or replicate very little.

I admin a DR site and replicate from production. Both HDFS and HIVE BDR jobs replicate nothing or very little at all. The kb/s throughput is 0%. Even a job with 17 small files doesn't replicate the display just spins. They don't fail they just don't do anything. I'm on CM 5.14.3 and CDH 5.13.3. The servers have plenty of free rseources, disk i/o and network are good.

Posts: 519
Topics: 14
Kudos: 92
Solutions: 45
Registered: ‎09-02-2016

Re: BDR jobs don't replicate or replicate very little.

@DanielWhite

 

In General, there could be so many reasons for these kind of issues, 

 

1. Both the source and destination clusters must have a Cloudera Enterprise license

 

https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_bdr_replication_intro.html#concep...

 

2.  Pls refer the below link to understand the supported/unsupported scenarios. Ex: for unsupported: The clusters use different Kerberos realms. etc..

 

https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_bdr_replication_intro.html#concep...

Posts: 1,035
Topics: 1
Kudos: 258
Solutions: 128
Registered: ‎04-22-2014

Re: BDR jobs don't replicate or replicate very little.

@DanielWhite,

 

Also, if a YARN job has launched, you can review the progress of the containers like any YARN job.  They may provide some insight into what is slow.

 

To help define where the problem is occurring, you can try creating a new replication schedule to replicate files to/from the same cluster.  This will help you see if the problem involves the other cluster.

Explorer
Posts: 21
Registered: ‎12-13-2016

Re: BDR jobs don't replicate or replicate very little.

The fix for this was to upgrade CDH from 5.5.1 to 5.13.3

Announcements