Member since
09-21-2021
4
Posts
0
Kudos Received
0
Solutions
09-25-2021
07:27 AM
@asish When hiveserver2 connect to multiple metastore, how does it know which metastore to use? Let's say, a user run "SHOW DATABASES;" using beeline and connected to hiveserver2. Does it return all DBs from all metastore? Another case, both metastore A and metastore B have the same database name, dbC. When user run "SHOW TABLES FROM dbC", will it return all tables from dbC in metastore A, B, or both?
... View more
09-25-2021
06:58 AM
@asish Ah, I see. That means I don't have to create/run new MySQL instance for each metastore. Is it possible for one hiveserver2 to connect to multiple metastore, so all user will use the same endpoint? or do I have to run another hiveserver2 for each metastore?
... View more
09-25-2021
06:42 AM
@smruti thanks for the reply. So, recently i found this Jira issue about adding catalog feature to Hive https://issues.apache.org/jira/browse/HIVE-18755 . It seems that with catalog, it is possible to create a database with the same name under different catalog. It's a new feature, I think. Added to the newest Hive (version 3). But, I can't find any doc for how to use it. Looking to the thrift file, there's a function to manage the catalog (create, update, delete, and get). But there's no function the get all databases under that catalog. Maybe because it's a new feature, so it's still under development. Or, maybe I'm misunderstood the purpose of this feature, or I missed the docs. I also want to ask, does any authentication and authorization tools like Kerberos or Ranger do not affect the creation of a database with the same name even though the user is different?
... View more
09-21-2021
08:22 PM
Is it possible for Hive Metastore to support multi-tenancy? I'm trying to build a multi-tenant feature in Hive Metastore. It means that there will be multiple user and each user can create databases, tables, etc. Also, the database or tables created by that user cannot be accessed by another users. I've thought maybe by providing an authentication to Hive will give a multi-tenancy feature. But after tried it with Kerberos, it seems that different users cannot create database with the same name, for example when user1 have a database named dbA then user2 cannot create database named dbA. I've not tried using authorization tools like Ranger, but it looks like it will not give me multi-tenancy feature (different user can create database with the same name). There are several solutions that I can think of, that is: - Each user have their own Metastore. It seems inefficient to have a lot of hive metastore, but it is the most straight forward solution to me. - Include username in database name. Database name will have this format: <username>_<database-name>. But I'm not so sure about this one. I've been searching about this topic but found very little information. The official Hive documentation doesn't even mention about it, or maybe I missed it (?). Or are there any other method/solution to implement multi-tenant feature in Hive Metastore? Thank you
... View more
Labels:
- Labels:
-
Apache Hive