Why/How is hive replication through BDR "single-threaded"

[#1]

I am testing BDR functionality and have not managed to create a working Hive replication job yet. Currently when running it I am getting 

Message: The remote command failed with error message: Another Hive replication command is already running for Database: MY_TABLE_NAME Table: . on service HIVE-2. 

I previously had Hive replication failing immediately because I had not specified the port (443) for the CM peering. 

What is causing this to fail immediately? I cannot see any logs apart from the above error message. 

I happen to be copying betwee two clusters within the same Cloudera Manager - but wont always be. 

Any ideas?

thanks

[#2]

Hello bgooley,

Thanks for your suggestion. I am re-reading the docs and I still think it tells me to add a CM peer any time I want to do a BDR replication, but I can accept that maybe my reading of the docs is wrong.

I have been re-trying my tests without a peered CM but was not able to improve the situation.

In the meantime we have taken a different track and started to use a new cluster as the target with a new Cloudera Manager and BDR seems to be working for that.

[#3]

@alexmc6,

A peer is normally only created if replicating from a cluster that has another Cloudera Manager.

If you have two clusters using the same Cloudera Manager, your replication schedule should let you choose which Hive server and cluster will be the source and target.

If that isn't working out, please show a screen shot of what selections are availavble in your replication schedule.

[#4]

@alexmc6,

I talked with the Cloudera engineering team and indeed, you are not supposed to create a peer from/to the same CM instance.  In fact, our UI blocks that unless you trick it by using a different hostname or ip address.

We will work on improving our documentaiton to explain how to replicate between clusters managed by the same Cloudera Manager.

If you have any follow-up questions, we're happy to help.

Ben

[#5]

Thanks for the information!