Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can we copy entire table or database with data and metadata through falcon ?

Re: Can we copy entire table or database with data and metadata through falcon ?

@Saurabh Kumar

Is your cluster kerberized ?

Re: Can we copy entire table or database with data and metadata through falcon ?

@Saurabh Kumar

jdbc:hive2://lxhdpmasttst002.lowes.com:10001/landing_db;auth=delegationToken: looks wrong to me. Could you share the cluster entity file of both source and target.

Re: Can we copy entire table or database with data and metadata through falcon ?

Rising Star

Shouldnt the uri look like jdbc:hive2://<host>:<port>/<db>;transportMode=http;httpPath=<http_endpoint> ?

Re: Can we copy entire table or database with data and metadata through falcon ?

Guru

@jramakrishnan:

Cluster entity:

Source Cluster :

<?xml version="1.0"?>

<cluster colo="USWestOregon" description="oregonHadoopCluster" name="hortonworksSourceCluster" xmlns="uri:falcon:cluster:0.1">

<interfaces>

<interface type="readonly" endpoint="hftp://lxhdpmasttst001.lowes.com:50070" version="2.3.4" />

<interface type="write" endpoint="hdfs://lxhdpmasttst001.lowes.com:8020" version="2.3.4" />

<interface type="execute" endpoint="lxhdpmasttst002.lowes.com:8050" version="2.3.4" />

<interface type="workflow" endpoint="http://lxhdpmasttst001.lowes.com:11000/oozie/" version="4.2.0" />

<interface type="registry" endpoint="thrift://lxhdpmasttst002.lowes.com:9083" version="1.2.1"/>

<interface type="messaging" endpoint="tcp://lxhdpmasttst001.lowes.com:61616?daemon=true" version="5.1.6" />

</interfaces>

<locations>

<location name="staging" path="/apps/falcon/hortonworksSourceCluster/staging" />

<location name="temp" path="/tmp" />

<location name="working" path="/apps/falcon/hortonworksSourceCluster/working" />

</locations>

</cluster>

Target Cluster:

<?xml version="1.0"?>

<cluster colo="USEastVirginia" description="virginiaHadoopCluster" name="hortonworksTargetCluster" xmlns="uri:falcon:cluster:0.1">

<interfaces>

<interface type="readonly" endpoint="hftp://lxhdpmastdev001.lowes.com:50070" version="2.3.4" />

<interface type="write" endpoint="hdfs://lxhdpmastdev001.lowes.com:8020" version="2.3.4" />

<interface type="execute" endpoint="lxhdpmastdev002.lowes.com:8050" version="2.3.4" />

<interface type="workflow" endpoint="http://lxhdpmastdev002.lowes.com:11000/oozie/" version="4.2.0" />

<interface type="registry" endpoint="thrift://lxhdpmastdev002.lowes.com:9083" version="1.2.1"/>

<interface type="messaging" endpoint="tcp://lxhdpmastdev002.lowes.com:61616?daemon=true" version="5.1.6" />

</interfaces>

<locations>

<location name="staging" path="/apps/falcon/hortonworksTargetCluster/staging" />

<location name="temp" path="/tmp" />

<location name="working" path="/apps/falcon/hortonworksTargetCluster/working" />

</locations>

</cluster>

Re: Can we copy entire table or database with data and metadata through falcon ?

Guru

@jramakrishnan:

no we don't have kerberized cluster but we are running our HS2 on port 10001.

is it because we are using http transport mode with HS2 using Hive Replication. does it work with https mode or not ?

Highlighted

Re: Can we copy entire table or database with data and metadata through falcon ?

@Saurabh Kumar

Are you able to make connection to beeline as below.

!connect jdbc:hive2://hiveserver.domain.com:10001/<db_name>;transportMode=http;httpPath=cliservic

e;

Re: Can we copy entire table or database with data and metadata through falcon ?

Guru

@jramakrishnan: Yes I am able to make connection through above connection string.

Re: Can we copy entire table or database with data and metadata through falcon ?

Guru

@jramakrishnan: Did you get anything ?

Re: Can we copy entire table or database with data and metadata through falcon ?