Member since
08-01-2017
3
Posts
2
Kudos Received
0
Solutions
08-02-2017
07:23 AM
1 Kudo
Thank you for your response. To clarify, we have purchased 2 Hadoop clusters: one for dev and test and the second for production. Each cluster will have 15 nodes (4 master, 9 worker, and 2 edge). My background is in Oracle, so I need further clarification on how to architect the Hadoop environment. (It is my understanding that in Hadoop, a "database namespace" is synonymous with "schema owner" in Oracle) We current have three Oracle databases: dwdev, dwtst, and dwprd and they are each on their own hardware server. Within each database, there is a schema owner named marketing which owns a set of tables, ie. customer_master, product_master, etc. If we want to simulate this in Hadoop, except that dev and test will exist on the same hardware cluster, how do we do that given that we do not want to change the existing schema owner and table names? So, in oracle, you have .,. Does this require having separate metastores?
... View more
08-01-2017
11:05 AM
We are moving our Oracle "landing" data into Hadoop. In Oracle we have three environments and three Oracle databases: dwdev, dwtest, and dwprod. The goal is to have three separate "landing" zones in Hadoop that will feed into each Oracle database, respectively, i.e. Hadoop dev feeds Oracle dwdev, etc. The dev and test hadoop environment will exist on a single physical hadoop cluster. How do we architect this? HDFS /<env>/data/<information_area>/<table_name> /dev/data/marketing/customer_master /test/data/marketing/customer_master HIVE database namespace (or schema_owner) = db_marketing table name = customer_master In DEV select * from db_marketing.customer_master would source from /dev/data/marketing/customer_master In TEST select * from db_marketing.customer_master would source from /test/data/marketing/customer_master Does this require multiple metastores? What is best practice for multiple environments on a single Hadoop cluster?
... View more
Labels:
- Labels:
-
Apache Hive
-
HDFS