Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Falcon job running on source instead of target

Falcon job running on source instead of target

New Contributor

Hi all,

I configured a Falcon Hive mirror to copy a table from a cluster to another as per this article:

https://community.hortonworks.com/content/kbentry/55382/hive-disaster-recovery-using-falcon.html

But when I try to execute the job, it is running on the source cluster, even if I specify the radio button to run on target.

This is on Falcon 0.6.1.2.3 and HDP 2.3.

Am I missing something?

Thanks,

Kiran

5 REPLIES 5

Re: Falcon job running on source instead of target

Contributor

Can you post xml version of your falcon job?

Re: Falcon job running on source instead of target

New Contributor

@Rahul Reddy

Here you go:

<?xml version="1.0" encoding="UTF-8"?> <process xmlns="uri:falcon:process:0.1" name="myHive"> <tags>_falcon_mirroring_type=HIVE</tags> <clusters> <cluster name="primaryCluster"> <validity start="2017-03-23T15:58Z" end="2017-03-23T16:30Z" /> </cluster> </clusters> <parallel>1</parallel> <order>LAST_ONLY</order> <frequency>days(20)</frequency> <timezone>GMT-06:00</timezone> <properties> <property name="oozie.wf.subworkflow.classpath.inheritance" value="true" /> <property name="distcpMaxMaps" value="1" /> <property name="distcpMapBandwidth" value="100" /> <property name="targetCluster" value="targetCluster" /> <property name="sourceCluster" value="primaryCluster" /> <property name="targetHiveServer2Uri" value="hive2://QAHiveServer2:10000" /> <property name="sourceHiveServer2Uri" value="hive2://ProdHiveServer2:10000" /> <property name="sourceStagingPath" value="/apps/falcon/primaryCluster/staging" /> <property name="targetStagingPath" value="/apps/falcon/backupCluster/staging" /> <property name="targetNN" value="hdfs://QANameNode:8020" /> <property name="sourceNN" value="hdfs://ProdNameNode:8020" /> <property name="sourceServicePrincipal" value="hive" /> <property name="targetServicePrincipal" value="hive" /> <property name="targetMetastoreUri" value="thrift://localhost:9083" /> <property name="sourceMetastoreUri" value="thrift://localhost:9083" /> <property name="targetHiveMetastoreKerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="sourceHiveMetastoreKerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="targetHive2KerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="sourceHive2KerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="sourceTable" value="globalsales" /> <property name="sourceDatabase" value="default" /> <property name="maxEvents" value="-1" /> <property name="replicationMaxMaps" value="5" /> <property name="clusterForJobRun" value="targetCluster" /> <property name="clusterForJobRunWriteEP" value="hdfs://QANameNode:8020" /> <property name="drJobName" value="myHive" /> <property name="drNotificationReceivers" value="kiran.erra@EMailID" /> </properties> <workflow name="falcon-dr-hive-workflow" engine="oozie" path="/apps/data-mirroring/workflows/hive-disaster-recovery-workflow.xml" lib="" /> <retry policy="periodic" delay="minutes(30)" attempts="3" /> <ACL owner="ambari-qa" group="users" permission="0755" /> </process>

Highlighted

Re: Falcon job running on source instead of target

Contributor

@kerra

Can you obtain the feed definition from Falcon CLI to validate the same.

$FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -definition

https://falcon.apache.org/FalconCLI.html

Also , you can check the job definition and job configuration from related oozie workflow.

Re: Falcon job running on source instead of target

Rising Star

Re: Falcon job running on source instead of target

New Contributor

Thanks Beverly. I'll check on that.