I restart HMS via Cloudera Manager. On the page of HMS/processes, the hive-site.xml listed under Configuration Files is what I meant runtime hive-site.xm, which has different contents than /etc/hive/conf/hive-site.xml. ... View more

I deployed Hive client configuration files and /etc/hive/conf/hive-site.xml timestamp is up to date. The file has defined hive.metastore.uris. However, the runtime hive-site.xml does not hve hive.metastore.uris. The stderr of HMS shows "Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore". Any ideas how to resolve this issue? ... View more

Bypass Hive Metastore Server is unchecked. I wonder why the runtime hive-site.xml has hive.metastore.local set to true. The server has 32GB RAM, but I don't think allocating more java heap size would help. It seems to have memory leak. ... View more

Thank you for your help. I was able to increase java heap size of hive metatstore server in bytes to 6GB. It just took longer (25 minutes) for the process to die. The usage of resident memory was up to 5.8GB. I notice that the runtime hive-site.xml has hive.metastore.local set to true and hive.metastore.uris is not there. We use Oracle Oracle as metastore. Is this normal? ... View more

Thanks for your help. I increased java heap size of hive metastore server in bytes to 6GB. It just took longer (25 minutes) for the process to die. The resident memory usage was 5.8GB.   I noticed that the runtime hive-site.xml has hive.metastore.local set to true and there is no hive.metastore.uris property. We are using Oracle as the data store. Is this normal? ... View more

I can't find any relevant heap size configuration for Hive Metastore Server. I did a search of heap size on Cloudera Manager page and two found related to Hive. Hive: Client Java Heap Size in Bytes Hive: Java Heap Size of HiveServer2 in Bytes Neither of them works. ... View more

I changed Hive: Client Java Heap size to 2GB, but the run time environment of Hive Metastore the process still shows 1GB. I need help how to change java heap size for metastore and make it effective. Thanks in advance. ... View more

Thanks. I found the the Resident Memory chart and the stderr. The processes tab only appears when the Hive Metastore Server is up, that is why I don't see it when the role is down. The resident memory usage reached 1.1GB before the role went down. This situation repeats every single time. Is there is a limit set for java heap size? I am pasting the end of stderr below. /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.3.0-cdh5.1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen-2.3.0-cdh5.1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/hadoop-sls-2.3.0-cdh5.1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-sls.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming-2.3.0-cdh5.1.0.jar:/usr/lib/hadoop-mapreduce/hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/hamcrest-core-1.3.jar:/usr/lib/hadoop-mapreduce/httpclient-4.2.5.jar:/usr/lib/hadoop-mapreduce/httpcore-4.2.5.jar:/usr/lib/hadoop-mapreduce/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop-mapreduce/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/jackson-xc-1.8.8.jar:/usr/lib/hadoop-mapreduce/jasper-compiler-5.5.23.jar:/usr/lib/hadoop-mapreduce/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-mapreduce/java-xmlbuilder-0.4.jar:/usr/lib/hadoop-mapreduce/jaxb-ap i-2.2.2.jar:/usr/lib/hadoop-mapreduce/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop-mapreduce/jersey-core-1.9.jar:/usr/lib/hadoop-mapreduce/jersey-json-1.9.jar:/usr/lib/hadoop-mapreduce/jersey-server-1.9.jar:/usr/lib/hadoop-mapreduce/jets3t-0.9.0.jar:/usr/lib/hadoop-mapreduce/jettison-1.1.jar:/usr/lib/hadoop-mapreduce/jetty-6.1.26.jar:/usr/lib/hadoop-mapreduce/jetty-util-6.1.26.jar:/usr/lib/hadoop-mapreduce/jsch-0.1.42.jar:/usr/lib/hadoop-mapreduce/jsp-api-2.1.jar:/usr/lib/hadoop-mapreduce/jsr305-1.3.9.jar:/usr/lib/hadoop-mapreduce/junit-4.11.jar:/usr/lib/hadoop-mapreduce/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/metrics-core-3.0.0.jar:/usr/lib/hadoop-mapreduce/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/protobuf-java-2.5.0.jar:/usr/lib/hadoop-mapreduce/servlet-api-2.5.jar:/usr/lib/hadoop-mapreduce/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/stax-api-1.0-2.jar:/usr/lib/hadoop-mapreduce/xmlenc-0.52.jar:/usr/lib/hadoop-mapreduce/xz-1.0.jar:/usr/lib/hadoop- mapreduce/zookeeper.jar:/usr/share/java/mysql-connector-java.jar:/usr/share/cmf/lib/postgresql-9.0-801.jdbc4.jar:/usr/share/java/oracle-connector-java.jar + [[ -n '' ]] + copy_hive_default /var/run/cloudera-scm-agent/process/454-hive-HIVEMETASTORE + [[ -z 5 ]] + '[' 1 -ne 1 ']' + [[ 5 == 3 ]] + [[ metastore == webhcat ]] + [[ metastore == updatelocation ]] + [[ metastore == create_metastore_tables ]] + [[ metastore == upgrade_metastore ]] + exec /usr/lib/hive/bin/hive --config /var/run/cloudera-scm-agent/process/454-hive-HIVEMETASTORE --service metastore -p 9083 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces 14/09/05 17:14:56 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative 14/09/05 17:14:57 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore. + kill -9 16129 ... View more

I followed the section of Configuring the Hive Metastore in the CDH 5.1 installaiton guide. I modified hive-site.xml because I did not see javax.jdo.option.ConnectionURL in it. There is no stderr log for Hive Metastore role. I can't find Java heap size charts either. During the installation, the Spark service can't start due to some data nodes are not started. After manually starting data nodes and Spark service, we manaully finish the rest of steps, like deploy client configuration file. Do you think that can be why javax.jdo.option.ConnectionURL is not in the hive-stie.xml? Any advice would be appreciated. ... View more