Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Metastore (Some basic questions)

Hive Metastore (Some basic questions)

New Contributor

I am new to Hadoop and Ecosystem.

 

I understand that Hive sits ontop of Hadoop, but confuguration wise, have some question

 

1. Does Hive metastore can be out of Hadoop ?, Let say separate box of my sql, and cofigure Hive metastore on that machine?

2. Is there anything like Hive Server and Hive Client ?

3. If yes for #2, where does Hive Server configured, and Where Hive client installed?

4. If i have my laptop, can i install hive client on my machine and connect to Hive Server? like SQL Server SSMS ?

 

Please help,

 

Thanks, Virul

4 REPLIES 4

Re: Hive Metastore (Some basic questions)

Champion

The below link covers in detail the different options and has wonderful and simple diagram on how the different components interact with the Hive Metastore (HMS).

 

1. Yes, the HMS does site outside (although the community is bring it into HBase to have it scale better) and in a regular RDBMS like MySQL.  I recommend putting it on a Master node as then it can be segrated and protected better.

2. There is HiveServer2 (HS2) and the Hive client.  Both access the metastore.  HS2 is much improved of the original HiveServer and this is how you should have most of your clients connect.  As the doc covers you can have it on the same host but that brings some security concerns.  I recommend having it on a edge/gateway host that can connect to the HMS.  The Hive client skips past HS2 and connects directly to HMS.  There are concerns around that like bypassing authorization, permanent UDFs created in it aren't available to HS2 clients, etc.

3. Let me know if #2 didn't cover this well.

4. You can install the Hive client and go in that way. Your laptop would need direct access to HMS.  Or you can connect through JDBC/ODBC to HS2 like you would most another DBs.

 

https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_metastore_configure.html

Re: Hive Metastore (Some basic questions)

Champion

@mbigelow Could you please explain me more on this statement "  The Hive client skips past HS2 and connects directly to HMS " -  Curious to know . are you refering to Custom user defined function in hive ? 

Re: Hive Metastore (Some basic questions)

Champion
@csguna

The Hive client connect to HMS directly it doesn't connect to HS2 at all. This why Sentry authorization doesn't work for it and why when you create a permanent UDF HS2 clients don't know about it until HS2 is restarted. It create the UDF in the metastore and until a restart (or reload command) HS2 isn't aware of it. If a HS2 client create a UDF then all HS2 client can use it right away.

A simple demonstration, open a hive connection using the cli client and then turn off HS2 or vice versa.

The image at this bookmark in the link makes it clear.

https://www.cloudera.com/documentation/enterprise/5-8-x/topics/cdh_ig_hive_metastore_configure.html#...

Re: Hive Metastore (Some basic questions)

Champion

1. Does Hive metastore can be out of Hadoop ?, Let say separate box of my sql, and cofigure Hive metastore on that machine?

 

Yes can certaintly configure Hive metastore outside Hadoop. 

You have 3 options. 

One is Embedded Metastore - Derby  -  Disadvantage is you cant have multi session 

Local Metastore  - You have Driver that Connects to Metastore

Remote Metastore - 

 

2. Is there anything like Hive Server and Hive Client ?

 

Hiveserver2  - Runs Hive as a Server exposing it to thrift Service , 

Main advanatge is authentication and Multi user concurrency 

 

Note  - You need Zookeeper for this type of configuration . 

 

 

 

3. If yes for #2, where does Hive Server configured, and Where Hive client installed?

 

there are different mechanism to connect  - thrift client , jdbc driver ( u can use beeline ) , odbc driver 

 

4 . Yes you can do that certainly , we have to make sure that we have all the configuration in hive-site.xml like hive -metastore uri , jdbc driver , hive zookeeper qurom , jdbc connection url  etc are set properly 

Don't have an account?
Coming from Hortonworks? Activate your account here