Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Falcon replication and mirroring between two Kerberized clusters?

Solved Go to solution

Falcon replication and mirroring between two Kerberized clusters?

New Contributor

Does Falcon support replication and mirroring for HDFS and Hive between two Kerberized clusters (Prod and DR)?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Falcon replication and mirroring between two Kerberized clusters?

Set

<property>
    <name>oozie.launcher.mapreduce.job.hdfs-servers</name>
    <value>${nameNode1},${nameNode2}</value>
</property>

As this documentation suggests: http://oozie.apache.org/docs/4.1.0/DG_DistCpActionExtension.html

5 REPLIES 5

Re: Falcon replication and mirroring between two Kerberized clusters?

Can you tell if both these env using same KDC server? DistCP (which is what is used by Falcon for data replication) supports copying data between two kerberos environments so its a matter of making sure that the clusters can communicate correctly.

Re: Falcon replication and mirroring between two Kerberized clusters?

New Contributor

Both clusters will be using either local KDC or AD Kerberos

Re: Falcon replication and mirroring between two Kerberized clusters?

Set

<property>
    <name>oozie.launcher.mapreduce.job.hdfs-servers</name>
    <value>${nameNode1},${nameNode2}</value>
</property>

As this documentation suggests: http://oozie.apache.org/docs/4.1.0/DG_DistCpActionExtension.html

Re: Falcon replication and mirroring between two Kerberized clusters?

Rising Star

Falcon will automatically add this property to oozie jobs in secure kerberized clusters. User does not have to add the property separately.

When defining a cluster entity, please make sure to add the following cluster property.

    <property name="dfs.namenode.kerberos.principal" value="nn/$my.internal@EXAMPLE.COM"/> 

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/configur...

Re: Falcon replication and mirroring between two Kerberized clusters?

Rising Star

Replication works between clusters using Kerberos since Dal (and some additional scenarios using HCat works from Dal-M10