HBase along with Phoenix is one of the most powerful NoSQL combinations. HBase/Phoenix capabilities allow users to host OLTPish workloads natively on Hadoop using HBase/Phoenix with all the goodness of HA and analytic benefits on a single platform (Ie Spark-hbase connector or Phoenix Hive storage handler). Often a requirement for HA implementations is a need for DR environment. Here I will describe a few common patterns and in no way is this the exhaustive HBase DR patterns. In my opinion, pattern 5 is the simplest to implement and provides operational ease & efficiency. Here are some of the high level replication and availability strategies with HBase/Phoenix HBASE provides High Availability within a cluster by managing region server failures transparently. HBASE provides various cross DC asynchronous replication schemes Master/Master replication topology Two clusters replicating all edits, bi-directionally to each other Master/Slave topology replication One cluster replicating all edits to second cluster Cyclic topology for replication A ring topology for clusters, replicating all edits in an acyclic manner Hub and spoke topology for replication A central cluster replicating all edits to multiple clusters in a uni-directional manner Using various topologies described above cross DC replication scheme can be setup as per desired architecture Pattern 1 Reads & Writes served by both clusters An implementation of client to provide for stickiness for writes/reads based on a session ID like concept needs to investigated Master/Master replication between clusters Bidirectional replication Replication post failover - recovery instrumented via Cyclic Replication Pattern 2 Reads served by both clusters Writes served by single cluster Master/Master replication between clusters Bidirectional replication Client will failover to secondary cluster Replication post failover - recovery instrumented via Cyclic Replication Pattern 3 Reads & Writes served by single cluster Master/Master replication between clusters Bidirectional replication Client will failover to secondary cluster Replication post failover - recovery instrumented via Cyclic Replication Pattern 4 Reads & Writes served by single cluster Master/Slave replication between clusters Unidirectional replication Client will failover to secondary cluster Manual resync required on ”primary” cluster due to unidirectional replication Pattern 5 Ingestion via NiFi Rest API Supports handling secure calls and round trip responses Push data to Kafka to democratize data to all apps interested in data set Secure Kafka topics via Apache Ranger NiFi dual ingest into N number of HBase/Phoenix clusters Enables in-sync data stores Operational ease NiFi back pressuring will handle any ODS downtime UI flow orchestration Data Governance built in via Data Provenance Event level linage Additional HBase Replication Documentation Monitor replication status https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/hbase-replication-monitoring-status.html Replication metrics https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/hbase-cluster-replication-metrics.html Replication Configuration options https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/hbase-cluster-repl-repl-config-options.html HBaseReplication Internals https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/hbase-replication-internals.html HBase Cluster Replication Details https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.0/bk_hadoop-ha/content/hbase-cluster-repl-details.html ... View more

A often requested feature has popped up in HDP 2.5, Phoenix namespace mapping AKA Phoenix schemas. This feature allows creation of schemas and development within a schema work space. This guide will walk through steps to enable this feature and how to use it. Use Case: Create 2 schemea named "schema1" and "scheama2" Within both schemas create table customer. Create another schema named "schema3". Drop schema "schema3" Note - A schema can only be dropped if it is empty (does not host any tables). 1. Through Ambari go to HBase. Select Advanced tab 2. Add a parameter to hbase-site.xml. To do this go to "Custom hbase-site" and click on "Add Property" 3. Add the property Key: phoenix.schema.isNamespaceMappingEnabled Value: true Click on Add button. A restart hbase service will be required. 4. Test the functionality, use sqlline.py. It is located under /usr/hdp/current/phoenix/bin Note - I am using /usr/hdp/2.5.0.0-1245/phoenix/bin since I want to prove to the audience I am in fact using HDP 2.5. 5. Create two schemeas named "schema1" and "schmea2" 6. Set work space to use schema1 and create table customer 7. Set work space to use schema2 and create table customer 8. Run a !table to view all schemas and tables 9. Create a schema "schema3" and drop "schema3" That's it! Simple and easy to use. Enjoy the new feature ... View more