Support Questions
Find answers, ask questions, and share your expertise

Running multiple hive (not Hive HA) in single cluster

Highlighted

Running multiple hive (not Hive HA) in single cluster

New Contributor

We are using CDH 5.15.0. We want to run multiple hive metastores/databases in order to run jobs for different environments (DEV and QA) in a single Cloudera cluster. For this, I came across the following link: https://community.cloudera.com/t5/Support-Questions/Config-multiple-Hive-servers-Metastores/m-p/2267...


Based on @ssubhas suggestion in above link, I am running the following command:

 

hive 'HIVE_CONF_DIR=/etc/hive2/conf.cloudera.hive/hive-site.xml /bin/hiveserver2 -hiveconf hive.metastore.uris=" " -hiveconf hive.log.dir=/var/log/hive2 -hiveconf hive.log.file=hiveserver2.log 1>/var/log/hive2/hiveserver2.log 2>/var/log/hive2/hiveserver2.log'


and get this output:


Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0

Logging initialized using configuration in jar:file:/opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/jars/hive-common-1.1.0-cdh5.15.0.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.


Now when I run following command, it shows a different set of databases. Seems it's connecting to the existing hive and ignoring above hive configuration dir (HIVE_CONF_DIR)

 

hive> show databases;

 

Also attaching /etc/hive2/conf.cloudera.hive/hive-site.xml. This is the only file i have in /etc/hive2/conf.cloudera.hive/ directory.

 

I will really appreciate any help on this.

5 REPLIES 5
Highlighted

Re: Running multiple hive (not Hive HA) in single cluster

Rising Star

Does this hiveserver2 managed by CDH cluster or independent of CDH ?

Highlighted

Re: Running multiple hive (not Hive HA) in single cluster

New Contributor

It is CDH managed.

 

$ ls -l /bin/hiveserver2
lrwxrwxrwx 1 root root 29 Oct 7 20:46 /bin/hiveserver2 -> /etc/alternatives/hiveserver2
$ ls -l /etc/alternatives/hiveserver2
lrwxrwxrwx 1 root root 66 Oct 7 20:46 /etc/alternatives/hiveserver2 -> /opt/cloudera/parcels/CDH-5.15.0-1.cdh5.15.0.p0.21/bin/hiveserver2

Highlighted

Re: Running multiple hive (not Hive HA) in single cluster

Rising Star

It wont work in CDH or HDP managed cluster.

 

You need to try in independent not managed by CDH or HDP as per Sindhu's comment

Re: Running multiple hive (not Hive HA) in single cluster

New Contributor

In Sindhu's comment, she mentioned that in Ambari-managed cluster it's not possible but in non-Ambari managed Hive process, it is possible. Please correct me if i am wrong.

Highlighted

Re: Running multiple hive (not Hive HA) in single cluster

Rising Star

She meant to say for non managed HDP/CDH.

Don't have an account?