Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Falcon Doesn't Mirror Data?

Solved Go to solution

Falcon Doesn't Mirror Data?

Expert Contributor

Hi,

I set up two HDP clusters and tried to use Falcon to mirror data from one cluster to another.

I could created two clusters and configured the mirror setting on Falcon UI without error. I could also see the mirroring MR job completed successfully. However, files do not get mirrored from source to target.

I am attaching my Falcon Mirror setting. Am I missing anything?

4662-falcon-mirror.png

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Falcon Doesn't Mirror Data?

Expert Contributor

Thanks @yzheng

The mirror instance is in healthy status.

After upgraded the target cluster to 2.3.4 and changed to job to run on source cluster, it works now.

8 REPLIES 8

Re: Falcon Doesn't Mirror Data?

Expert Contributor

@yjiang What is the HDP version you are using, for which you have seen that Falcon does not mirror data.

Re: Falcon Doesn't Mirror Data?

Expert Contributor

@peeyush That is the target cluster running HDP 2.3.4.7-4. The source cluster is HDP 2.4.2.0

Re: Falcon Doesn't Mirror Data?

@Rahul Pathakcan you help here ?

Re: Falcon Doesn't Mirror Data?

New Contributor

@yjiang Could you check the mirror instance status? You can run CLI command

bin/falcon instance -list -type process -name [your-mirror-name]

If it is failed or killed, could you check Oozie site on error details?

Re: Falcon Doesn't Mirror Data?

Expert Contributor

Thanks @yzheng

The mirror instance is in healthy status.

After upgraded the target cluster to 2.3.4 and changed to job to run on source cluster, it works now.

Re: Falcon Doesn't Mirror Data?

New Contributor

@yjiang Great!

Re: Falcon Doesn't Mirror Data?

There is a bug in hdfs replication workflow FALCON-189 causing this. As a quick fix, edit drSourceDir to include source FS, so that it become: ${drSourceClusterFS}${drSourceDir} as discussed in the Jira. The file to fix is

/usr/hdp/current/falcon-server/data-mirroring/workflows/hdfs-replication-workflow.xml

on your Falcon server. Then upload the file to /apps/data-mirroring/workflows in HDFS. Restart Falcon and retry to run mirroring from the target cluster.

Highlighted

Re: Falcon Doesn't Mirror Data?

Cloudera Employee

Upload the hdfs-replication-workflow.xml into hdfs will resolve this problem.

i.e. create a hdfs directory /apps/data-mirroring/workflows and upload it

Don't have an account?
Coming from Hortonworks? Activate your account here