I provisioned an HDP 2.6 cluster using HD Cloud. When I first start up the cluster I get a critical Ambari alert for HiveServer2 interactive. I tried restarting zookeeper and hive, restarting the master host, but I still get the error.
Ambari is unable to make a connection to the interactive port of Hiveserver2:
ExecutionFailed: Execution of '! beeline -u 'jdbc:hive2://ip-10-0-68-150.us-west-1.compute.internal:10500/;transportMode=binary' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL'' returned 1. Error: Could not open client transport with JDBC Uri: jdbc:hive2://ip-10-0-68-150.us-west-1.compute.internal:10500/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)
Error: Could not open client transport with JDBC Uri: jdbc:hive2://ip-10-0-68-150.us-west-1.compute.internal:10500/;transportMode=binary: java.net.ConnectException: Connection refused (Connection refused) (state=08S01,code=0)
)Hiveserver2 interactive log has the following error indicating an issue with Zookeeper:
2017-04-29 18:38:05,175 INFO [main-SendThread(ip-10-0-68-150.us-west-1.compute.internal:2181)]: zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(1019)) - Opening socket connection to server ip-10-0-68-150.us-west-1.compute.internal/10.0.68.150:2181. Will not attempt to authenticate using SASL (unknown error)
The zookeeper log contains the following error messages:
2017-04-29 18:40:05,624 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbae5b75a000a type:delete cxid:0x338 zxid:0x20e txntype:-1 reqpath:n/a Error Path:/registry/users/hive/services/org-apache-slider/llap0 Error:KeeperErrorCode = Directory not empty for /registry/users/hive/services/org-apache-slider/llap0
2017-04-29 18:40:05,992 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d00008 type:create cxid:0x1 zxid:0x213 txntype:-1 reqpath:n/a Error Path:/services Error:KeeperErrorCode = NodeExists for /services
2017-04-29 18:40:05,998 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d00008 type:create cxid:0x2 zxid:0x214 txntype:-1 reqpath:n/a Error Path:/services/slider Error:KeeperErrorCode = NodeExists for /services/slider
2017-04-29 18:40:05,999 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d00008 type:create cxid:0x3 zxid:0x215 txntype:-1 reqpath:n/a Error Path:/services/slider/users Error:KeeperErrorCode = NodeExists for /services/slider/users
2017-04-29 18:40:06,000 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d00008 type:create cxid:0x4 zxid:0x216 txntype:-1 reqpath:n/a Error Path:/services/slider/users/hive Error:KeeperErrorCode = NodeExists for /services/slider/users/hive
2017-04-29 18:40:06,043 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d00009 type:delete cxid:0x1 zxid:0x218 txntype:-1 reqpath:n/a Error Path:/registry/users/hive/services/org-apache-slider/llap0 Error:KeeperErrorCode = NoNode for /registry/users/hive/services/org-apache-slider/llap0
2017-04-29 18:40:21,698 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d0000a type:delete cxid:0x1 zxid:0x21d txntype:-1 reqpath:n/a Error Path:/registry/users/hive/services/org-apache-slider/llap0 Error:KeeperErrorCode = NoNode for /registry/users/hive/services/org-apache-slider/llap0
2017-04-29 18:40:21,728 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d0000a type:create cxid:0x2 zxid:0x21e txntype:-1 reqpath:n/a Error Path:/registry/users/hive/services/org-apache-slider Error:KeeperErrorCode = NodeExists for /registry/users/hive/services/org-apache-slider
2017-04-29 18:41:00,709 - INFO [ProcessThread(sid:0 cport:-1)::PrepRequestProcessor@645] - Got user-level KeeperException when processing sessionid:0x15bbb0169d0000f type:create cxid:0x1 zxid:0x22e txntype:-1 reqpath:n/a Error Path:/hiveserver2-hive2 Error:KeeperErrorCode = NodeExists for /hiveserver2-hive2
Any help would be much appreciated.
