Support Questions
Find answers, ask questions, and share your expertise

Falcon job running on source instead of target

Falcon job running on source instead of target

Explorer

Hi all,

I configured a Falcon Hive mirror to copy a table from a cluster to another as per this article:

https://community.hortonworks.com/content/kbentry/55382/hive-disaster-recovery-using-falcon.html

But when I try to execute the job, it is running on the source cluster, even if I specify the radio button to run on target.

This is on Falcon 0.6.1.2.3 and HDP 2.3.

Am I missing something?

Thanks,

Kiran

5 REPLIES 5

Re: Falcon job running on source instead of target

Contributor

Can you post xml version of your falcon job?

Re: Falcon job running on source instead of target

Explorer

@Rahul Reddy

Here you go:

<?xml version="1.0" encoding="UTF-8"?> <process xmlns="uri:falcon:process:0.1" name="myHive"> <tags>_falcon_mirroring_type=HIVE</tags> <clusters> <cluster name="primaryCluster"> <validity start="2017-03-23T15:58Z" end="2017-03-23T16:30Z" /> </cluster> </clusters> <parallel>1</parallel> <order>LAST_ONLY</order> <frequency>days(20)</frequency> <timezone>GMT-06:00</timezone> <properties> <property name="oozie.wf.subworkflow.classpath.inheritance" value="true" /> <property name="distcpMaxMaps" value="1" /> <property name="distcpMapBandwidth" value="100" /> <property name="targetCluster" value="targetCluster" /> <property name="sourceCluster" value="primaryCluster" /> <property name="targetHiveServer2Uri" value="hive2://QAHiveServer2:10000" /> <property name="sourceHiveServer2Uri" value="hive2://ProdHiveServer2:10000" /> <property name="sourceStagingPath" value="/apps/falcon/primaryCluster/staging" /> <property name="targetStagingPath" value="/apps/falcon/backupCluster/staging" /> <property name="targetNN" value="hdfs://QANameNode:8020" /> <property name="sourceNN" value="hdfs://ProdNameNode:8020" /> <property name="sourceServicePrincipal" value="hive" /> <property name="targetServicePrincipal" value="hive" /> <property name="targetMetastoreUri" value="thrift://localhost:9083" /> <property name="sourceMetastoreUri" value="thrift://localhost:9083" /> <property name="targetHiveMetastoreKerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="sourceHiveMetastoreKerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="targetHive2KerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="sourceHive2KerberosPrincipal" value="hive/_HOST@EXAMPLE.COM" /> <property name="sourceTable" value="globalsales" /> <property name="sourceDatabase" value="default" /> <property name="maxEvents" value="-1" /> <property name="replicationMaxMaps" value="5" /> <property name="clusterForJobRun" value="targetCluster" /> <property name="clusterForJobRunWriteEP" value="hdfs://QANameNode:8020" /> <property name="drJobName" value="myHive" /> <property name="drNotificationReceivers" value="kiran.erra@EMailID" /> </properties> <workflow name="falcon-dr-hive-workflow" engine="oozie" path="/apps/data-mirroring/workflows/hive-disaster-recovery-workflow.xml" lib="" /> <retry policy="periodic" delay="minutes(30)" attempts="3" /> <ACL owner="ambari-qa" group="users" permission="0755" /> </process>

Re: Falcon job running on source instead of target

Contributor

@kerra

Can you obtain the feed definition from Falcon CLI to validate the same.

$FALCON_HOME/bin/falcon entity -type [cluster|datasource|feed|process] -name <<name>> -definition

https://falcon.apache.org/FalconCLI.html

Also , you can check the job definition and job configuration from related oozie workflow.

Re: Falcon job running on source instead of target

Rising Star

Re: Falcon job running on source instead of target

Explorer

Thanks Beverly. I'll check on that.