Solved
HDP 3.1 hiveserver2 not starting
Hi there, I'm facing this problem for 2 days now and maybe you guys can find it.
I'm not the only one facing this issue but none of the given solutions fixed my problem.
Hive metastore is running and hiveserver2 is not saying connection refused I will put my exact error messages here too.
Connection failed on host localhost:10000 (Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/HIVE/package/alerts/alert_hive_thrift_port.py", line 204, in execute
ldap_password=ldap_password)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/hive_check.py", line 84, in check_thrift_port_sasl
timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE,
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
ExecutionFailed: Execution of 'beeline -n hive -u 'jdbc:hive2://localhost:10000/;transportMode=binary' -e ';' 2>&1 | awk '{print}' | grep -i -e 'Connected to:' -e 'Transaction isolation:'' returned 1.
)
and it gives this when I am trying to start hiveserver2:
2020-01-30 10:56:14,981 - Execution of 'cat /var/run/hive/hive-server.pid 1>/tmp/tmp8y1m5V 2>/tmp/tmpRyZFzp' returned 1. cat: /var/run/hive/hive-server.pid: No such file or directory
2020-01-30 10:56:14,982 - get_user_call_output returned (1, u'', u'cat: /var/run/hive/hive-server.pid: No such file or directory')
2020-01-30 10:56:14,982 - call['ambari-sudo.sh su hive -l -s /bin/bash -c 'hive --config /usr/hdp/current/hive-server2/conf/ --service metatool -listFSRoot' 2>/dev/null | grep hdfs:// | cut -f1,2,3 -d '/' | grep -v 'hdfs://localhost:8020' | head -1'] {}
2020-01-30 10:56:26,621 - call returned (0, '')
2020-01-30 10:56:26,621 - Execute['/var/lib/ambari-agent/tmp/start_hiveserver2_script /var/log/hive/hive-server2.out /var/log/hive/hive-server2.err /var/run/hive/hive-server.pid /usr/hdp/current/hive-server2/conf/ /etc/tez/conf'] {'environment': {'HIVE_BIN': 'hive', 'JAVA_HOME': u'/usr/jdk64/jdk1.8.0_112', 'HADOOP_HOME': u'/usr/hdp/current/hadoop-client'}, 'not_if': 'ls /var/run/hive/hive-server.pid >/dev/null 2>&1 && ps -p >/dev/null 2>&1', 'user': 'hive', 'path': [u'/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/var/lib/ambari-agent:/usr/hdp/current/hive-server2/bin:/usr/hdp/3.1.4.0-315/hadoop/bin']}
2020-01-30 10:56:26,652 - Execute['/usr/jdk64/jdk1.8.0_112/bin/java -cp /usr/lib/ambari-agent/DBConnectionVerification.jar:/usr/hdp/current/hive-server2/lib/mysql-connector-java.jar org.apache.ambari.server.DBConnectionVerification 'jdbc:mysql://novalinq-stoeptegel/hive?createDatabaseIfNotExist=true' hive [PROTECTED] com.mysql.jdbc.Driver'] {'path': ['/usr/sbin:/sbin:/usr/local/bin:/bin:/usr/bin'], 'tries': 5, 'try_sleep': 10}
2020-01-30 10:56:27,434 - call['/usr/hdp/current/zookeeper-client/bin/zkCli.sh -server localhost:2181 ls /hiveserver2test | grep 'serverUri=''] {}
2020-01-30 10:56:28,121 - call returned (1, 'Node does not exist: /hiveserver2test')
2020-01-30 10:56:28,121 - Will retry 29 time(s), caught exception: ZooKeeper node /hiveserver2test is not ready yet. Sleeping for 10 sec(s)
I've also tried it with Zookeeper node /hiveserver2 (without 'test') and that did not work either.
In the hiveserver2.log file I can not find anything particular, can you? (how do I attach a file?)
I have tried the following (without succes):
- Re-installed Hive service
- Checked hive-site.xml (everything seems right)
- Checked if the ports that Hive is using are free (they are so far I know)
- Checked in the ZkCli.sh if I could find hiveserver2 (can not find it, it is not there)
- Tried DEBUG mode for Hive, did not get any other messages (bit weird).
I am running HDP 3.1.4.0 single node cluster (not yet for production, first getting everything started)
Thank you in advance!
Dewi
