HBASE/FALCON replication

[#1]

I am trying to setup Falcon to setup HBASE replication.

So I tried to create a Falcon cluster as outlined in :

https://hortonworks.com/hadoop-tutorial/create-falcon-cluster/

But after creating the Falcon user and logging into Ambari as Falcon user, I found the screen empty - without the Your Views options as listed in above url under sub heading : Preparing HDFS directories.

But if I log out and then re-login as admin then I can see the views for Yarn, Hive and Tez.

Am I missing something?

Appreciate the feedback.

[#2]

It is all good now. Thanks.

[#3]

In Creating a Cluster Entity, I am not sure what values to use.

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_data_governance/content/section_creating...

What are the values to be entered in the Interfaces and Properties sections?

The Hortonworks documentation lacks specific instructions.

[#4]

The interfaces are required component. I think when you enter into that page, it will give you sample value, i.e. hftp://sandbox.hortonworks.com:50070, you then need to change the url to you own setting if not on the sandbox. Detail information on these setting, you can find at https://falcon.apache.org/EntitySpecification.html.

The properties are optional, you can define your own properties. You can check in the link I listed above as well.

[#5]

Frank thanks. I found all the required values and clicked on Save. It is still giving some error but I am not able to see that the error is. If I click on Notifications I can see some old errors but not able to scroll down to see the latest error. If I click on the scroll bar the Notification window simply closes. Any idea where the physical logs of falcon are held?

[#6]

It finally came through after a long long time. I thought there was an error but after more than an hour I got a create cluster success message! Ok, next step is to create the Falcon backup cluster. My HDP backup cluster is actually another separate physical cluster. So should I create the Falcon backup cluster on the primary or the backup cluster. Appreciate the feedback.

[#7]

I had setup Falcon mirroring based on this document

https://hortonworks.com/hadoop-tutorial/mirroring-datasets-between-hadoop-clusters-with-apache-falco...

The one single difference is that unlike the example in the document which seems to be using a single hdfs cluster, I am using two different clusters - each with its set of nodes.

At the outset after running the mirror job it is NOT working for me.

Some initial questions are :

1. After creating a Falcon Cluster, how do we view it(s) definition? I had created two falcon clusters - one for the primary and another for the target. I would like to see the information falcon holds for them.

2. After executing a Falcon job how do we check the logs to see what it did and where it failed?

Appreciate the insights.

[#8]

Hello, any feedback on the Falcon issues? I have configured the Falcon clusters and also the job. But replication is not happening. I enabled Ozzie log as shown in the Hortonworks site (including restarting Oozie) but still I can see the log as disabled in the job error log. Not sure why.

Also in Falcon UI if I click on the job I can see one or two instances (?) of the job running, while the rest are WAITING. So to restart the job I just have to Suspend and Resume?

So when will the RUNNING jobs end and when will the WAITING jobs start?

Appreciate the feedback.

[#9]

Could you open a new thread, so that somebody else will chime in if I am not available?

[#10]

See if these docs help: http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-views/content/ch_using_falcon_view... and http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_data-movement-and-integration/content/ind....