Created on 12-07-202212:17 AM - edited 12-13-202212:59 AM
In this article, we will walk through the steps to integrate Data Engineering Experience & Operational DataBase Experience within a few clicks with an example of reading & writing data into Phoenix tables using PySpark script.
As you know Cloudera Operational Database supports both Traditional relational schema and Evolutionary schema with the capability to transact on petabytes of structured and unstructured data generated by businesses today further details are attached.
Whereas Cloudera Data Engineering streamlines and operationalizes data pipelines securely at any scale to increase efficiency and accelerate time to value further details attached.
Example: Customer 360 - Cloudera Operational Database will store customer profiling data for business applications and the data is modeled & prepared with Cloudera Data Engineering. Both capabilities come with High Availability, Scalability, Mission Critical Performance, and Flexibility for handling any type of data.
In our demo, we are leveraging CDP Public Cloud but the same experience/set-up can be done on CDP Private Cloud Data Services:
A cluster Environment should be provisioned. Refer to this link for the same.
Cloudera Operational Database should be provisioned. Refer tothis link for the same.
Cloudera Data Engineering Virtual cluster should be provisioned. Refer tothis link for the same.
As a first step we need to make Cloudera Operation Database experience accessible to the Cloudera Data Engineering experience, In order to do that, we need to have an Hbase-site.xml config file and it can be easily obtained following the steps below:
Assuming the Cluster Environment is ready as described in the Pre-Req steps, go to the Go ToHome Page of the Cloudera Operational Database.
Click on the 'DataBase'you have created as described in the Pre-Reqs (In our example it's demo-database).
Under the "Connect" tab of the database, copy theHBase Client Configuration URL field. Following is the screenshot for the same:
Use $curl and the URL copied in step-3 on your terminal to download the Hbase Config file as shown below: