Member since
09-11-2015
27
Posts
10
Kudos Received
9
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
814 | 07-07-2016 07:35 PM | |
1000 | 06-22-2016 03:33 PM | |
1696 | 06-21-2016 03:29 PM | |
887 | 06-13-2016 08:09 PM | |
2175 | 06-06-2016 05:02 PM |
07-11-2016
04:14 PM
Without the date specification, the file will be overwritten with each replication, so you will have no history. If this is acceptable, simply specify the location as /hdfsDataLocation.
... View more
07-07-2016
07:35 PM
Unfortunately, today the answer is no.
... View more
06-22-2016
03:33 PM
2 Kudos
Before setting up Falcon Mirroring, the Hive databases and tables should be "seeded" to the backup cluster using Hive's export/import table.
Dumping and seeding the MySQL repository is not a recommended method. When setting up the Mirror in Falcon, it is best to mirror entire databases, not individual tables.
That way any new tables created on the source cluster will automatically be mirrored onto the backup cluster. We recommend running the Mirror on the backup (target) cluster so the mirroring workload has less impact on your production cluster.
... View more
06-21-2016
04:18 PM
1 Kudo
I'm working with Falcon 0.8, but Avro replication was fixed in the version shipped with 2.3.0. The documentation is quite lacking at this point, an issue we are working to resolve now. My apologies. Are you encountering an issue?
... View more
06-21-2016
04:12 PM
For high availability with Solr, the best practice is probably using SolrCloud. I believe with SolrCloud, you let Solr handle the replication by creating additional shards. The Solr docs have more info (http://archive.apache.org/dist/lucene/solr/ref-guide/apache-solr-ref-guide-5.2.pdf).
... View more
06-21-2016
03:56 PM
Performance is going to be largely predicated by the speed of the network / distance between the data centers. Falcon uses distcp under the covers.
... View more
06-21-2016
03:34 PM
1 Kudo
Falcon can be used to replicate Solr transaction logs and index. If the index is active, replication may fail and be automatically retried. Therefore, it's best to schedule replication for off-peak periods.
... View more
06-21-2016
03:29 PM
Yes. In HDP 2.3.0 and higher, Falcon supports replication of Avro files.
... View more
06-13-2016
08:09 PM
Oozie cannot directly reference Falcon objects. To use Oozie with Falcon, create an Input Feed and Output feed and a Falcon Oozie process. Falcon will then generate the Oozie workflow XML.
... View more
06-06-2016
05:02 PM
Oozie examples are bundled within the Oozie distribution in the oozie-examples.tar.gz file. Expand the file and an examples directory will be created in the local file system. Copy the directory to the user HOME directory in HDFS. The examples provide Pig and Java examples as well as a local Oozie example that can be used for testing. Modifying the scripts to work with other actions such as Hive 2 or shell can be achieved by referring to the documentation for those actions.
... View more