Reply
Explorer
Posts: 20
Registered: ‎05-02-2017

Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

Hi 

 

I am totally lost , where do I have to place the MYSQL JDBC driver when my cluster is managed by cloduera manager . also I am in the middle of Cloudera manager installation where I mentioned to HIVE as to have a custom database . 

 

I have downloaded and installed mysql database 5.7.8 

created hive user and database metastore 

gave all the permission to the user.

 

I am stuck up when it comes to placing the JDBC jar where do i place it so that Cloduera manager will recogonize other it complains saying JDBC driver not found in the host-name 

Between also I dont see any usr/lib/hive directory being created . 

i only see /etc/hive/conf 

any help is highy appreciable 

Cloudera Employee
Posts: 116
Registered: ‎03-23-2015

Re: Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

Hi,

Do you mean MySQL connector or JDBC driver?

You should put the connectors under /usr/share/java.

For JDBC drivers, you can just add them to the classpath.

Cloudera does not use /usr/lib/hive, rather, all server side configurations are under /var/run/cloudera-scm-agent/process, have a look at this directory on your server host.

/etc/hive directories are only used by Client applications, like beeline or Hive CLI, not HiveServer2 or HMS.

Hope that clarifies your concern.
Explorer
Posts: 20
Registered: ‎05-02-2017

Re: Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

Hi @EricL

Sorry for the confusion , I am trying to make HIVE use mysql database as Metastore . 

So I guess i need to say that it is mysql-connector-java.jar . 

So having said that I should place that .jar file under /user/share/java / to make Cloduera manager recgonize and pick it up not complain . Correct me if I am wrong. 

Cloudera Employee
Posts: 15
Registered: ‎01-07-2014

Re: Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

Hi MattPatt,

 

For Hive, it is the Hive Metastore server (and not Cloudera Manager) that needs access to the JDBC driver to talk to the database.

 

As mentioned in the documentation, JDBC driver needs to be installed on the Cloudera Manager Server host, as well as hosts to which you assign the Hive Metastore Server, Activity Monitor, Reports Manager, Hue Server, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server roles.

 

See https://www.cloudera.com/documentation/enterprise/latest/topics/cm_ig_mysql.html#cmig_topic_5_5_3

for details on the location.

 

thanks,

Mohit

Cloudera Employee
Posts: 116
Registered: ‎03-23-2015

Re: Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

Hi Matt

As Mohit mentioned, you can have a look at the doc provided, it has more details.

To answer your question, yes, it is under /usr/share/java, but it is for HMS and other services that uses the connector to recognize it, not CM.
Explorer
Posts: 20
Registered: ‎05-02-2017

Re: Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

@EricL @mohit @mbigelow  @saranvisa

Sorry for the brutal late response. 

Could you clarify the below 

 

1 . if I use The JDK that  is included in the Cloudera Manager 5 repositories

Do i have to manually add JAVA_HOME in any of the Cloudera manager configuration files 

if so where is the location of those files ? 

 

2 . Thanks for the clarification on mysql driver on usr/share/java for HMS

 

3 . Thrid , I Would like to know if /usr/share/java is the location for HMS then could you tell me where should those mysql jdbc jar be placed for Cloudera manager to recogonize because i am plaining to have all those cloudera manager service like Activity Monitor, Reports Manager,  Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server to be in mysql database not use the postgres database that ships along

 

Thanks for any information its really going to help in my prod box. 

 

Cloudera Employee
Posts: 116
Registered: ‎03-23-2015

Re: Hive Metastore configuration - Cloduera Manager - Cluster -JDBC not picking up

Hi Matt,

1. I am not too sure, but I believe you do not need to as CM will configure the cluster automatically. Again, I am not 100% sure, maybe others can help

2. ^_^

3. /usr/share/java is the default location for mysql connector jar file, so if for connector jar, I believe all you need is put under there on the host that need it and it will be picked up by other components.

Cheers
Announcements