Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive Server2 service getting stopped

Highlighted

Hive Server2 service getting stopped

Contributor

i have seen a regular behavior of hive server2 getting stopped while we are using hive from Qliksense or multiple query from Zeppelin. I am using HDP-2.4.2.0 .I am getting the following error.

Connection failed on host XXX:10000 (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_thrift_port.py", line 200, in execute check_command_timeout=int(check_command_timeout)) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hive_check.py", line 74, in check_thrift_port_sasl timeout=check_command_timeout) File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__ self.env.run() File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action) File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action() File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 238, in action_run tries=self.resource.tries, try_sleep=self.resource.try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call raise Fail(err_msg) Fail: Execution of '! beeline -u 'jdbc:hive2://XXXXX:10000/;transportMode=binary' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'' returned 1. stdin: is not a tty stdin: is not a tty Error: Could not open client transport with JDBC Uri: jdbc:hive2://xxxx:10000/;transportMode=binary: java.net.ConnectException: Connection refused (state=08S01,code=0) Error: Could not open client transport with JDBC Uri: jdbc:hive2://lxdb4231.dc.corp.telstra.com:10000/;transportMode=binary: java.net.ConnectException: Connection refused (state=08S01,code=0)

7 REPLIES 7
Highlighted

Re: Hive Server2 service getting stopped

Hi @bandhu gupta

Can you please share the hiveserver2 logs?

Highlighted

Re: Hive Server2 service getting stopped

Contributor

This is the hive error shown below.

21545:2016-09-21 18:34:04,785 ERROR [HiveServer2-Handler-Pool: Thread-78]: metadata.Hive (Hive.java:getTable(1121)) - Table XXXXX not found: default.XXXXX table not found 21622:2016-09-21 18:34:05,004 ERROR [HiveServer2-Handler-Pool: Thread-78]: hdfs.KeyProviderCache (KeyProviderCache.java:createKeyProviderURI(87)) - Could not find uri with key [dfs.encryption.key.provider.uri] to create a keyProvider !!

Highlighted

Re: Hive Server2 service getting stopped

Expert Contributor

A highly possible cause can be HiveServer2 running out of memory.

The defaults from recent Ambari versions are quite low: 512MB of Java heap.

Check /var/log/hive/hiveserver2.log and /var/log/hive/hive-server2.out for any indication of such an issue.

And increase HiveServer2 Heap using Ambari from the default 512MB.

2048MB would be a start:

7829-screen-shot-2016-09-21-at-141832.png

Highlighted

Re: Hive Server2 service getting stopped

Contributor

I have increased the heap size to 1638 MB , Metastore heap size to 8052 Mb , Client heap size to 1024 mb but still i am getting this random behavior . Please suggest.

Highlighted

Re: Hive Server2 service getting stopped

Super Guru
@bandhu gupta

Please share hiveserver2 logs. It is likely a memory issue. Look for "Out of memory" error in your logs. Should be in the stack trace towards the end. Also, please share your hiveserver 2 config. There are also memory settings in hive-env.sh but you can set these from Ambari.

Highlighted

Re: Hive Server2 service getting stopped

Contributor

<property> <name>hive.security.metastore.authorization.manager</name> <value>org.apache.hadoop.hive.ql.security.authorization.StorageBasedAuthorizationProvider</value> </property> <property> <name>hive.server2.allow.user.substitution</name> <value>true</value> </property> <property> <name>hive.server2.authentication</name> <value>NONE</value> </property> <property> <name>hive.server2.authentication.spnego.keytab</name> <value>HTTP/_HOST@EXAMPLE.COM</value> </property> <property> <name>hive.server2.authentication.spnego.principal</name> <value>/etc/security/keytabs/spnego.service.keytab</value> </property> <property> <name>hive.server2.enable.doAs</name> <value>false</value> </property> <property> <name>hive.server2.logging.operation.enabled</name> <value>true</value> </property> <property> <name>hive.server2.logging.operation.log.location</name> <value>/tmp/hive/operation_logs</value> </property> <property> <name>hive.server2.support.dynamic.service.discovery</name> <value>true</value> </property> <property> <name>hive.server2.table.type.mapping</name> <value>CLASSIC</value> </property> <property> <name>hive.server2.tez.default.queues</name> <value>default</value> </property> <property> <name>hive.server2.tez.initialize.default.sessions</name> <value>false</value> </property> <property> <name>hive.server2.tez.sessions.per.default.queue</name> <value>1</value> </property> <property> <name>hive.server2.thrift.http.path</name> <value>cliservice</value> </property> <property> <name>hive.server2.thrift.http.port</name> <value>10001</value> </property> <property> <name>hive.server2.thrift.max.worker.threads</name> <value>500</value> </property> <property> <name>hive.server2.thrift.port</name> <value>10000</value> </property> <property> <name>hive.server2.thrift.sasl.qop</name> <value>auth</value> </property> <property> <name>hive.server2.transport.mode</name> <value>binary</value> </property> <property> <name>hive.server2.use.SSL</name> <value>false</value> </property> <property> <name>hive.server2.zookeeper.namespace</name> <value>hiveserver2</value> </property>

Re: Hive Server2 service getting stopped

Contributor

did you find out the cause of this issue. I'm seeing the same problem

Don't have an account?
Coming from Hortonworks? Activate your account here