Configuring Hive

[#1]

I am using Cloudera Express 5.0.2 (#297 built by jenkins on 20140606-2222 git: 80907df78ba6b50c21a598f0caff8b00685d5961) with the CDH 5.1.2-1.cdh5.1.2.p0.3 parcel on an Ubuntu 12.04.4 LTS machine in a 4-node cluster.

I want to get rid of the following Hive warning:

14/09/09 16:31:36 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

and I'm therefore now looking at how to configure the Hive metastore.  If I go to the Hive configuration in my Cloudera Manager UI, and search for 'hive.metastore.uris', this configuration property is not found and I don't see a way to add it.

Furthermore, if I search for the file hive-site.xml, I find multiple versions of this file:

root@sandy-quad-1:~# find / -name hive-site.xml
/run/cloudera-scm-agent/process/456-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/455-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/454-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/453-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/452-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/431-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/423-deploy-client-config/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/407-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/406-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/403-hive-metastore-create-tables/hive-site.xml
/run/cloudera-scm-agent/process/394-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/393-deploy-client-config/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/363-impala-CATALOGSERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/362-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/361-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/331-impala-CATALOGSERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/330-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/329-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/305-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/304-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/303-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/296-impala-CATALOGSERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/295-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/294-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/269-impala-CATALOGSERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/267-deploy-client-config/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/247-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/244-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/243-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/216-hue-HUE_SERVER/hive-conf/hive-site.xml
/run/cloudera-scm-agent/process/207-hive-HIVEMETASTORE/hive-site.xml
/run/cloudera-scm-agent/process/206-hive-HIVESERVER2/hive-site.xml
/run/cloudera-scm-agent/process/204-hue-HUE_SERVER/hive-conf/hive-site.xml
/opt/cloudera/parcels/CDH-5.1.2-1.cdh5.1.2.p0.3/etc/hive/conf.dist/hive-site.xml
/etc/hive/conf.cloudera.hive1/hive-site.xml
/etc/hive/conf.cloudera.hive/hive-site.xml

 and I don't know which one is the one I need to change.

My questions are:

1. Is it possible to add configuration options that are not there in my Hive config in the Cloudera Manager UI?
2. What is the main hive-site.xml config file that I need to change, and what services do I need to restart in order for these changes to take effect?

Thanks for your help,

Bart

[#2 · Accepted solution]

Hi Bart,

There's not really a problem here. Hive emits that warning, but we've found that in some cases setting metastore.local will avoid bugs (specifically when running on postgresql). So better to have the warning than to have the bug.

This might help explain the process directory where you're finding all of those hive-site.xml files:
http://blog.cloudera.com/blog/2013/07/how-does-cloudera-manager-work/

CM will automatically generate the metastore uris property based on the configured Hive Metastore Server role's host and port. Unless you have "Bypass Hive Metastore Server" selected, CM will emit the metastore uris property in all client configuration (/etc/hive/conf/hive-site.xml) and all dependent services (impala, hue, etc) that need to talk to Hive. The hive deprecation warning is thrown whether or not the metastore uris are configured.

So you should just ignore this error.

Thanks,
Darren