Member since
01-27-2016
46
Posts
40
Kudos Received
4
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
875 | 10-24-2016 05:39 PM | |
807 | 03-30-2016 06:02 PM | |
904 | 02-28-2016 04:37 PM | |
1808 | 02-07-2016 07:57 AM |
10-24-2016
05:39 PM
I figured out that I was unable to do the Hive Service Check whenever I uninstalled Atlas (which is recommended for updating to HDP 2.5). I solved this problem by uninstalling Atlas right before the Update.
... View more
10-23-2016
06:08 PM
As I would like to upgrade from hdp 2.4 to 2.5 I am running through service checks of all my services. Even though Hive Services are fully operational and green whenever I run the Hive Service Check I get the following Error Log: stderr: /var/lib/ambari-agent/data/errors-998.txt Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py", line 191, in <module>
HiveServiceCheck().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py", line 93, in service_check
hcat_service_check()
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_service_check.py", line 65, in hcat_service_check
logoutput=True)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 71, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 93, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 141, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 294, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'env JAVA_HOME=/usr/lib/jvm/java /var/lib/ambari-agent/tmp/hcatSmoke.sh hcatsmokeida8c06c00_date522316 prepare true' returned 12. WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more stdout: /var/lib/ambari-agent/data/output-998.txt 2016-10-23 17:52:53,644 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-10-23 17:52:53,700 - call['ambari-python-wrap /usr/bin/hdp-select status hive-server2'] {'timeout': 20}
2016-10-23 17:52:53,727 - call returned (0, 'hive-server2 - 2.4.0.0-169')
2016-10-23 17:52:53,728 - Stack Feature Version Info: stack_version=2.4, version=None, current_cluster_version=2.4.0.0-169 -> 2.4
2016-10-23 17:52:53,738 - Running Hive Server checks
2016-10-23 17:52:53,739 - --------------------------
2016-10-23 17:52:53,741 - Server Address List : ['sandbox.hortonworks.com'], Port : 10000, SSL KeyStore : None
2016-10-23 17:52:53,741 - Waiting for the Hive Server to start...
2016-10-23 17:52:53,742 - Execute['! beeline -u 'jdbc:hive2://sandbox.hortonworks.com:10000/;transportMode=binary' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL''] {'path': ['/bin/', '/usr/bin/', '/usr/lib/hive/bin/', '/usr/sbin/'], 'user': 'ambari-qa', 'timeout': 30}
2016-10-23 17:52:58,502 - Successfully connected to sandbox.hortonworks.com on port 10000
2016-10-23 17:52:58,502 - Successfully stayed connected to 'Hive Server' on host: sandbox.hortonworks.com and port 10000 after 4.76081609726 seconds
2016-10-23 17:52:58,502 - Running HCAT checks
2016-10-23 17:52:58,502 - -------------------
2016-10-23 17:52:58,504 - checked_call['hostid'] {}
2016-10-23 17:52:58,508 - checked_call returned (0, 'a8c06c00')
2016-10-23 17:52:58,509 - File['/var/lib/ambari-agent/tmp/hcatSmoke.sh'] {'content': StaticFile('hcatSmoke.sh'), 'mode': 0755}
2016-10-23 17:52:58,510 - Execute['env JAVA_HOME=/usr/lib/jvm/java /var/lib/ambari-agent/tmp/hcatSmoke.sh hcatsmokeida8c06c00_date522316 prepare true'] {'logoutput': True, 'path': ['/usr/sbin', '/usr/local/bin', '/bin', '/usr/bin', '/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/lib64/qt-3.3/bin:/usr/lib/jvm/java/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/hdp/current/falcon-client/bin:/usr/hdp/current/hadoop-mapreduce-historyserver/bin:/usr/hdp/current/oozie-client/bin:/usr/hdp/current/falcon-server/bin:/usr/hdp/current/hadoop-yarn-client/bin:/usr/hdp/current/oozie-server/bin:/usr/hdp/current/flume-client/bin:/usr/hdp/current/hadoop-yarn-nodemanager/bin:/usr/hdp/current/pig-client/bin:/usr/hdp/current/flume-server/bin:/usr/hdp/current/hadoop-yarn-resourcemanager/bin:/usr/hdp/current/slider-client/bin:/usr/hdp/current/hadoop-client/bin:/usr/hdp/current/hadoop-yarn-timelineserver/bin:/usr/hdp/current/sqoop-client/bin:/usr/hdp/current/hadoop-hdfs-client/bin:/usr/hdp/current/hbase-client/bin:/usr/hdp/current/sqoop-server/bin:/usr/hdp/current/hadoop-hdfs-datanode/bin:/usr/hdp/current/hbase-master/bin:/usr/hdp/current/storm-client/bin:/usr/hdp/current/hadoop-hdfs-journalnode/bin:/usr/hdp/current/hbase-regionserver/bin:/usr/hdp/current/storm-nimbus/bin:/usr/hdp/current/hadoop-hdfs-namenode/bin:/usr/hdp/current/hive-client/bin:/usr/hdp/current/storm-supervisor/bin:/usr/hdp/current/hadoop-hdfs-nfs3/bin:/usr/hdp/current/hive-metastore/bin:/usr/hdp/current/zookeeper-client/bin:/usr/hdp/current/hadoop-hdfs-portmap/bin:/usr/hdp/current/hive-server2/bin:/usr/hdp/current/zookeeper-server/bin:/usr/hdp/current/hadoop-hdfs-secondarynamenode/bin:/usr/hdp/current/hive-webhcat/bin:/usr/hdp/current/hadoop-mapreduce-client/bin:/usr/hdp/current/knox-server/bin:/usr/hdp/current/hadoop-client/sbin:/usr/hdp/current/hadoop-hdfs-nfs3/sbin:/usr/hdp/current/hadoop-yarn-client/sbin:/usr/hdp/current/hadoop-hdfs-client/sbin:/usr/hdp/current/hadoop-hdfs-portmap/sbin:/usr/hdp/current/hadoop-yarn-nodemanager/sbin:/usr/hdp/current/hadoop-hdfs-datanode/sbin:/usr/hdp/current/hadoop-hdfs-secondarynamenode/sbin:/usr/hdp/current/hadoop-yarn-resourcemanager/sbin:/usr/hdp/current/hadoop-hdfs-journalnode/sbin:/usr/hdp/current/hadoop-mapreduce-client/sbin:/usr/hdp/current/hadoop-yarn-timelineserver/sbin:/usr/hdp/current/hadoop-hdfs-namenode/sbin:/usr/hdp/current/hadoop-mapreduce-historyserver/sbin:/usr/hdp/current/hive-webhcat/sbin:/root/bin:/var/lib/ambari-agent:/usr/hdp/current/hive-client/bin:/usr/hdp/current/hadoop-client/bin'], 'tries': 3, 'user': 'ambari-qa', 'try_sleep': 5}
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
2016-10-23 17:53:26,357 - Retrying after 5 seconds. Reason: Execution of 'env JAVA_HOME=/usr/lib/jvm/java /var/lib/ambari-agent/tmp/hcatSmoke.sh hcatsmokeida8c06c00_date522316 prepare true' returned 12. WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
2016-10-23 17:54:04,367 - Retrying after 5 seconds. Reason: Execution of 'env JAVA_HOME=/usr/lib/jvm/java /var/lib/ambari-agent/tmp/hcatSmoke.sh hcatsmokeida8c06c00_date522316 prepare true' returned 12. WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hive.hcatalog.cli.HCatDriver.run(HCatDriver.java:43)
at org.apache.hive.hcatalog.cli.HCatCli.processCmd(HCatCli.java:291)
at org.apache.hive.hcatalog.cli.HCatCli.processLine(HCatCli.java:245)
at org.apache.hive.hcatalog.cli.HCatCli.main(HCatCli.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
... 24 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:76)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
... 31 more
Command failed after 1 tries
... View more
Labels:
- Labels:
-
Apache Hive
03-30-2016
06:02 PM
1 Kudo
As I am using vmware I found a straight forward approach (link below) without doing the virtuabox conversion mentioned in the link above: https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1006371
... View more
03-30-2016
12:08 PM
2 Kudos
Dear Experts, as I was running out of space from my hortonworks sandbox, I was adding addtional 20GB to my vmware volume. The assigned process in vmware took some time but went through successfully. Unfortunately I cannot see the additional diskspace in the vmware in the / 'mountpoint' When I execute a ----------------------------------------- [root@sandbox ~]# df -a Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/vg_sandbox-lv_root 44717136 42355568 83376 100% / proc 0 0 0 - /proc sysfs 0 0 0 - /sys devpts 0 0 0 - /dev/pts tmpfs 12313492 76 12313416 1% /dev/shm /dev/sda1 487652 30249 431803 7% /boot none 0 0 0 - /proc/sys/fs/binfmt_misc vmware-vmblock 0 0 0 - /var/run/vmblock-fuse gvfs-fuse-daemon 0 0 0 - /root/.gvfs ------------------------------------------------ Can you please enlighten me on how I can extend the storage to the "vg_sandbox-lv_root". Thanks and br, Rainer
... View more
03-24-2016
06:44 PM
Will do, thanks ... me constantly improving 🙂
... View more
03-23-2016
08:02 PM
Worked like a charm 🙂 Thanks for your support!
... View more
03-23-2016
04:58 PM
4 Kudos
Dear Experts, I created a quite complex zeppelin notebook and accidentally returned a rather big result set for one of the queries (that I had to abort). Now when I reopen the notebook in the browser, zeppelin takes ages and breaks after a while (I assume my browsers buffers are running over ...) Any Idea on how I can clear/reset the zeppelin result sets without accessing/entering the notebook? Any other idea on how I can bring back my notebook? Thanks and br, Rainer
... View more
Labels:
- Labels:
-
Apache Zeppelin
02-28-2016
04:47 PM
1 Kudo
Works, thanks ... thought I tried it already ... 🙂
... View more
02-28-2016
04:37 PM
1 Kudo
This can be closed ... I had 2 HiveServer2 running on different notes as part of a previously failed hive install ... and this totally confused ranger and falcon ... after removing hive and reinstalling it everything is fine now ... thanks for your support!
... View more
02-28-2016
04:33 PM
2 Kudos
Using Hortonworks Sandbox 2.3.2 Running Listener Port on 9092 (see last statement in below log file of kafka-server-start.sh ---- [2016-02-28 16:20:33,691] INFO Client environment:java.class.path=:/usr/lib/ambari-metrics-kafka-sink/ambari-metrics-kafka-sink.jar:/usr/lib/ambari-metrics-kafka-sink/lib/commons-codec-1.2.jar:/usr/lib/ambari-metrics-kafka-sink/lib/jackson-xc-1.9.13.jar:/usr/lib/ambari-metrics-kafka-sink/lib/commons-httpclient-3.1.jar:/usr/lib/ambari-metrics-kafka-sink/lib/jackson-mapper-asl-1.9.13.jar:/usr/lib/ambari-metrics-kafka-sink/lib/commons-lang-2.6.jar:/usr/lib/ambari-metrics-kafka-sink/lib/jackson-core-asl-1.9.13.jar:/usr/lib/ambari-metrics-kafka-sink/lib/commons-logging-1.1.1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../core/build/dependant-libs-2.10.5*/*.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../examples/build/libs//kafka-examples*.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../contrib/hadoop-consumer/build/libs//kafka-hadoop-consumer*.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../contrib/hadoop-producer/build/libs//kafka-hadoop-producer*.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../clients/build/libs/kafka-clients*.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/bcprov-jdk16-1.46.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/commons-collections-3.2.1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/commons-configuration-1.10.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/commons-io-2.4.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/commons-lang-2.6.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/commons-logging-1.2.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/eclipselink-2.5.2-M1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/gson-2.2.4.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/guava-11.0.2.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/hadoop-auth-2.7.1.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/hadoop-common-2.7.1.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/httpclient-4.2.5.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/httpcore-4.2.5.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/httpmime-4.2.5.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jackson-core-asl-1.9.13.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jackson-jaxrs-1.8.1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jackson-jaxrs-1.8.3.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jackson-mapper-asl-1.9.13.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jackson-xc-1.8.3.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/javax.persistence-2.1.0.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jersey-bundle-1.17.1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/jopt-simple-3.2.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka_2.10-0.8.2.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka_2.10-0.8.2.2.3.2.0-2950-javadoc.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka_2.10-0.8.2.2.3.2.0-2950-scaladoc.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka_2.10-0.8.2.2.3.2.0-2950-sources.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka_2.10-0.8.2.2.3.2.0-2950-test.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka-clients-0.8.2.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka-ganglia-0.8.2.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/kafka-ganglia-0.8.2.2.3.2.0-2950-javadoc.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/log4j-1.2.16.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/lz4-1.2.0.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/metrics-core-2.2.0.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/metrics-ganglia-2.2.0.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/noggit-0.6.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/ojdbc6.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/ranger-kafka-plugin-0.5.0.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/ranger-plugins-audit-0.5.0.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/ranger-plugins-common-0.5.0.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/ranger-plugins-cred-0.5.0.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/ranger_solrj-0.5.0.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/scala-library-2.10.4.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/slf4j-api-1.7.6.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/slf4j-log4j12-1.6.1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/snappy-java-1.1.1.6.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/zkclient-0.5-1.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/zookeeper-3.4.6.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../libs/zookeeper.jar:/usr/hdp/2.3.2.0-2950/kafka/bin/../core/build/libs/kafka_2.10*.jar (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:os.version=2.6.32-573.7.1.el6.x86_64 (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,691] INFO Client environment:user.dir=/usr/hdp/2.3.2.0-2950/kafka (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,692] INFO Initiating client connection, connectString=sandbox.hortonworks.com:2181 sessionTimeout=30000 watcher=org.I0Itec.zkclient.ZkClient@44a8253a (org.apache.zookeeper.ZooKeeper) [2016-02-28 16:20:33,707] INFO Opening socket connection to server sandbox.hortonworks.com/192.168.0.109:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2016-02-28 16:20:33,713] INFO Socket connection established to sandbox.hortonworks.com/192.168.0.109:2181, initiating session (org.apache.zookeeper.ClientCnxn) [2016-02-28 16:20:33,719] INFO Session establishment complete on server sandbox.hortonworks.com/192.168.0.109:2181, sessionid = 0x153280fd58f002c, negotiated timeout = 30000 (org.apache.zookeeper.ClientCnxn) [2016-02-28 16:20:33,720] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient) [2016-02-28 16:20:33,942] INFO Loading logs. (kafka.log.LogManager) [2016-02-28 16:20:33,977] INFO Completed load of log test-0 with log end offset 0 (kafka.log.Log) [2016-02-28 16:20:33,986] INFO Completed load of log test1-0 with log end offset 0 (kafka.log.Log) [2016-02-28 16:20:33,989] INFO Logs loading complete. (kafka.log.LogManager) [2016-02-28 16:20:33,989] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager) [2016-02-28 16:20:33,991] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager) [2016-02-28 16:20:34,019] INFO Awaiting socket connections on sandbox.hortonworks.com:9092. (kafka.network.Acceptor)---- Created Topic test successfully Whenever I try to push messages to the Kafka topic through e.g. console-producer "kafka-console-producer.sh --broker-list localhost:9092 --topic test" I get the following error messages: ------- [root@sandbox kafka]# bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test t [2016-02-28 16:32:17,945] WARN Fetching topic metadata with correlation id 0 for topics [Set(test)] from broker [BrokerEndPoint(0,localhost,9092)] failed (kafka.client.ClientUtils$) java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:120) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) at kafka.producer.SyncProducer.send(SyncProducer.scala:115) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82) at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:68) at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:89) at kafka.utils.Logging$class.swallowError(Logging.scala:106) at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51) at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:68) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45) [2016-02-28 16:32:17,951] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed (kafka.utils.CoreUtils$) kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73) at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82) at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:68) at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:89) at kafka.utils.Logging$class.swallowError(Logging.scala:106) at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51) at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:68) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45) Caused by: java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:120) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) at kafka.producer.SyncProducer.send(SyncProducer.scala:115) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) ... 12 more [2016-02-28 16:32:17,955] WARN Fetching topic metadata with correlation id 1 for topics [Set(test)] from broker [BrokerEndPoint(0,localhost,9092)] failed (kafka.client.ClientUtils$) java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:120) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) at kafka.producer.SyncProducer.send(SyncProducer.scala:115) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82) at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49) at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:188) at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:152) at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:151) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:151) at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:96) at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:73) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45) [2016-02-28 16:32:17,961] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed (kafka.producer.async.DefaultEventHandler) [2016-02-28 16:32:18,063] WARN Fetching topic metadata with correlation id 2 for topics [Set(test)] from broker [BrokerEndPoint(0,localhost,9092)] failed (kafka.client.ClientUtils$) java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:120) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) at kafka.producer.SyncProducer.send(SyncProducer.scala:115) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82) at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:79) at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:89) at kafka.utils.Logging$class.swallowError(Logging.scala:106) at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51) at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:79) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45) [2016-02-28 16:32:18,064] ERROR fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed (kafka.utils.CoreUtils$) kafka.common.KafkaException: fetching topic metadata for topics [Set(test)] from broker [ArrayBuffer(BrokerEndPoint(0,localhost,9092))] failed at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:73) at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82) at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:79) at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:89) at kafka.utils.Logging$class.swallowError(Logging.scala:106) at kafka.utils.CoreUtils$.swallowError(CoreUtils.scala:51) at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:79) at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:88) at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:68) at scala.collection.immutable.Stream.foreach(Stream.scala:547) at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:67) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45) Caused by: java.nio.channels.ClosedChannelException at kafka.network.BlockingChannel.send(BlockingChannel.scala:120) at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75) at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74) at kafka.producer.SyncProducer.send(SyncProducer.scala:115) at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:59) ... 12 more ------------ Any Idea on what I am doing wrong? Thanks, Rainer
... View more
Labels:
- Labels:
-
Apache Kafka
02-23-2016
06:07 PM
1 Kudo
btw I am using a Postgres DB for oozie if that matters by any means ...
... View more
02-23-2016
06:05 PM
1 Kudo
No, unfortunately falcon never worked, oozie though is up and running ...
... View more
02-23-2016
06:03 PM
1 Kudo
Did a delete of the Falcon service already several times (through curl commands) .... delete worked but after reinstall no change ... any other advise? btw ... all other services are running well by now 🙂
... View more
02-23-2016
05:33 PM
2 Kudos
Dear Experts, I am trying to start Falcon on my cluster (HDP-2.3.4.0-3485, Ambari 2.2), but unfortunately it doesn't start up I get the following message in the ambari error log: ----- Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py", line 165, in <module>
FalconServer().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
File "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py", line 46, in start
self.configure(env, upgrade_type=upgrade_type)
File "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py", line 41, in configure
falcon('server', action='config', upgrade_type=upgrade_type)
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py", line 86, in falcon
owner = params.falcon_user)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 154, in __init__
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 158, in run
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/__init__.py", line 94, in find_provider
ImportError: No module named properties_file ----- Checking the /var/log/falcon directory for logs the directory unfortunately is empty ... Any hint is appreciated! Thanks & br, Rainer
... View more
Labels:
- Labels:
-
Apache Falcon
02-07-2016
07:57 AM
Dear All, I found the solution to the problem ... zookeeper has to be installed on the nodes the supervisors are running on ... why that is? I don't know ... thanks for your support!
... View more
02-06-2016
09:55 PM
1 Kudo
I am getting into it ... 🙂
... View more
02-06-2016
09:55 PM
1 Kudo
http://hortonworks.com/hadoop-tutorial/simulating-transporting-realtime-events-stream-apache-kafka/#section_4
... View more
02-06-2016
09:52 PM
1 Kudo
storm.zookeeper.servers in the storm config is set to my three servers running zookeeper (as per Ambaris default) so unfortunately this isn't the solution 😞
... View more
02-06-2016
08:31 PM
1 Kudo
Thanks Neeraj, though I don't understand the following: my storm nimbus is installed and running on a node where zookeeper is running on default port 2181. As I am using the storm topology from the tutorial (and I assume the topology is running from the nimbus node) does that mean the zookeeper address is somewhat hardcoded in the topology? ... I deployed the topology though from a node where zookeeper is not running (if that matters) ... As I am just starting with Storm playing around I might just have a lack of understanding for how things working together, so sorry for that!
... View more
02-06-2016
06:54 PM
1 Kudo
Dear All, just trying to run the tutorial and everything works fine until I start the Kafka Producer.
My Setup: HDP-2.3.4.0-3485 on a 3N Yarn Setup I am getting the following error in the k-event-processor-3-1454777987-worker-6700.log
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:java.io.tmpdir=/tmp
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:java.compiler=<NA>
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.name=Linux
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.arch=amd64
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:os.version=3.10.0-229.14.1.el7.x86_64
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.name=storm
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.home=/home/storm
2016-02-06 17:28:05.220 o.a.z.ZooKeeper [INFO] Client environment:user.dir=/home/storm
2016-02-06 17:28:05.221 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=.....:2181,....:2181,....:2181, sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@209d0f1c
2016-02-06 17:28:05.283 o.a.c.f.i.CuratorFrameworkImpl [INFO] Starting
2016-02-06 17:28:05.286 o.a.z.ClientCnxn [INFO] Opening socket connection to server .....:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.286 o.a.z.ZooKeeper [INFO] Initiating client connection, connectString=localhost sessionTimeout=20000 watcher=org.apache.curator.ConnectionState@33745cd
2016-02-06 17:28:05.287 o.a.z.ClientCnxn [INFO] Socket connection established to .....:2181, initiating session
2016-02-06 17:28:05.318 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.319 o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_60]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_60]
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) ~[zookeeper-3.4.6.2.3.4.0-3485.jar:3.4.6-3485--1]
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125) [zookeeper-3.4.6.2.3.4.0-3485.jar:3.4.6-3485--1]
2016-02-06 17:28:05.340 o.a.z.ClientCnxn [INFO] Session establishment complete on server hdp1n1.aye1vpcdev/10.0.202.157:2181, sessionid = 0x152b6fb9a45005e, negotiated timeout = 20000
2016-02-06 17:28:05.345 o.a.c.f.s.ConnectionStateManager [INFO] State change: CONNECTED
2016-02-06 17:28:05.434 o.a.z.ClientCnxn [INFO] Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2016-02-06 17:28:05.435 o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
Any Idea what I am doing wrong?
... View more
Labels:
- Labels:
-
Apache Storm
02-05-2016
05:31 PM
1 Kudo
After a cluster restart it worked ... will continue to trace why that happened ...
... View more
02-05-2016
04:01 PM
1 Kudo
Dear All, I am facing some strange behaviour. My setup: 3 Node Yarn Cluster (8GB Memory each Node / CentOS7 / AWS Installation) When firing up a simple M/R Job through Pig, the application started with 2 containers but is "pending" getting the 3rd Container from the Node Manager. When I look at the 3 Nodes in Unix I see the following: My Mappers are setup to consume 1,5 GB Containers, My Reducers to consume 2GB As what I can see immediately, is that there obviously is no free space available for starting the 3rd container, but there is lots of space available in the buff/cache that I was hoping is used by yarn too for firing up containers. Am I missing something? Br, Rainer
... View more
- Tags:
- Hadoop Core
- YARN
Labels:
- Labels:
-
Apache YARN
02-03-2016
12:43 PM
1 Kudo
Thanks Neeraj, that worked. If I am now creating a table in hive, it has the owner "admin" and group "hdfs" on hdfs. As what I understand, my Ambari-Server is initialized by user "root" and group "root" What are the mechanics behind this user transfer and how can I manipulate it e.g. if want to have another owner/group than "admin/hdfs" on hdfs for my freshly created table? best regards, Rainer
... View more
02-03-2016
10:25 AM
1 Kudo
Dear All, whenever I launch the hive view I get the following error: ----------------
H060 Unable to open Hive session: org.apache.thrift.protocol.TProtocolException: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive from IP 10.0.202.157:13:12, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:266, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:202, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:402, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:297, org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1253, org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1238, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1142, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:617, java.lang.Thread:run:Thread.java:745, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive from IP 10.0.202.157:21:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:83, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:422, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1657, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy20:open::-1, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:258, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive from IP 10.0.202.157:26:5, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:494, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:137, sun.reflect.GeneratedMethodAccessor11:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:497, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:Unauthorized connection for super-user: hive from IP 10.0.202.157:45:19, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1358, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:229, com.sun.proxy.$Proxy15:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:771, sun.reflect.GeneratedMethodAccessor7:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:497, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:252, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:104, com.sun.proxy.$Proxy16:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:2116, org.apache.hadoop.hdfs.DistributedFileSystem$22:doCall:DistributedFileSystem.java:1315, org.apache.hadoop.hdfs.DistributedFileSystem$22:doCall:DistributedFileSystem.java:1311, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1311, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1424, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:568, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:526, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:480], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive from IP 10.0.202.157), serverProtocolVersion:null) ----------------------- I have the following proxy users in custom core-site: hadoop.proxyuser.hcat.groups = * hadoop.proxyuser.hcat.hosts = *hadoop.proxyuser.hdfs.groups = *hadoop.proxyuser.hdfs.hosts = *hadoop.proxyuser.hive.groups = *hadoop.proxyuser.hive.hosts = *hadoop.proxyuser.root.groups = *hadoop.proxyuser.root.hosts = *Also I granted permission to use the Hive View to "admin" in Ambari.What am I missing?Any support is appreciated!br,Rainer
... View more
Labels:
- Labels:
-
Apache Ambari
-
Apache Hive
01-30-2016
05:22 PM
1 Kudo
OK ... here is where I am ... I changed my primary group of user nifi to nifi ... i stopped nifi through "nifi.sh stop" I cleared /var/log/nifi and /var/run/nifi when I start nifi now from ambari there is nothing in stderr and I get the following feedback in ambari: stderr: /var/lib/ambari-agent/data/errors-1806.txt None stdout: /var/lib/ambari-agent/data/output-1806.txt 2016-01-30 16:31:57,093 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485
2016-01-30 16:31:57,094 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0
2016-01-30 16:31:57,094 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-01-30 16:31:57,115 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '')
2016-01-30 16:31:57,115 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-01-30 16:31:57,135 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0')
2016-01-30 16:31:57,136 - Ensuring that hadoop has the correct symlink structure
2016-01-30 16:31:57,136 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-01-30 16:31:57,244 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485
2016-01-30 16:31:57,244 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0
2016-01-30 16:31:57,245 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-01-30 16:31:57,265 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '')
2016-01-30 16:31:57,266 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-01-30 16:31:57,286 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0')
2016-01-30 16:31:57,287 - Ensuring that hadoop has the correct symlink structure
2016-01-30 16:31:57,287 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-01-30 16:31:57,288 - Group['hadoop'] {}
2016-01-30 16:31:57,289 - Group['nifi'] {}
2016-01-30 16:31:57,289 - Group['users'] {}
2016-01-30 16:31:57,289 - User['zookeeper'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,290 - User['ams'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,290 - User['ambari-qa'] {'gid': 'hadoop', 'groups': [u'users']}
2016-01-30 16:31:57,291 - User['kafka'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,291 - User['hdfs'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,292 - User['yarn'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,292 - User['nifi'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,293 - User['mapred'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,293 - User['hbase'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 16:31:57,294 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-30 16:31:57,295 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2016-01-30 16:31:57,299 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2016-01-30 16:31:57,299 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2016-01-30 16:31:57,300 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-30 16:31:57,300 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2016-01-30 16:31:57,304 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2016-01-30 16:31:57,304 - Group['hdfs'] {'ignore_failures': False}
2016-01-30 16:31:57,305 - User['hdfs'] {'ignore_failures': False, 'groups': [u'hadoop', u'hdfs']}
2016-01-30 16:31:57,305 - Directory['/etc/hadoop'] {'mode': 0755}
2016-01-30 16:31:57,317 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-01-30 16:31:57,317 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777}
2016-01-30 16:31:57,328 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2016-01-30 16:31:57,335 - Skipping Execute[('setenforce', '0')] due to only_if
2016-01-30 16:31:57,335 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2016-01-30 16:31:57,337 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'}
2016-01-30 16:31:57,337 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'}
2016-01-30 16:31:57,341 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2016-01-30 16:31:57,342 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2016-01-30 16:31:57,343 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2016-01-30 16:31:57,350 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2016-01-30 16:31:57,350 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2016-01-30 16:31:57,354 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2016-01-30 16:31:57,357 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2016-01-30 16:31:57,536 - File['/opt/nifi-1.1.1.0-12/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 16:31:57,539 - File['/opt/nifi-1.1.1.0-12/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 16:31:57,543 - File['/opt/nifi-1.1.1.0-12/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 16:31:57,543 - Execute['echo pid file /var/run/nifi/nifi.pid'] {}
2016-01-30 16:31:57,546 - Execute['echo JAVA_HOME=/usr/jdk64/jdk1.8.0_60'] {}
2016-01-30 16:31:57,549 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_60;/opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2016-01-30 16:32:00,606 - Execute['cat /opt/nifi-1.1.1.0-12/bin/nifi.pid | grep pid | sed 's/pid=\(\.*\)/\1/' > /var/run/nifi/nifi.pid'] {}
2016-01-30 16:32:00,625 - Execute['chown nifi:nifi /var/run/nifi/nifi.pid'] {} Still though the service goes down after some green blinking ... when I am going to bash there is also no nifi service running ... in /var/log/nifi there are 4 log files nifi-app.log nifi-bootstrap.log nifi-setup.log nifi-user.log --------------------------------------------------- [nifi@nifi1n1 nifi]$ cat /var/log/nifi/* 2016-01-30 17:10:40,677 INFO [main] org.apache.nifi.bootstrap.RunNiFi No Bootstrap Notification Services configured. 2016-01-30 17:10:40,679 INFO [main] org.apache.nifi.bootstrap.Command Apache NiFi is not running Java home: /usr/jdk64/jdk1.8.0_60 NiFi home: /opt/nifi-1.1.1.0-12 Bootstrap Config File: /opt/nifi-1.1.1.0-12/conf/bootstrap.conf [nifi@nifi1n1 nifi]$ ---------------------------------------------------------- /var/run/nifi Directory is empty Access Rights are set appropriately for /var/log/nifi and /var/run/nifi [nifi@nifi1n1 log]$ ls -lisa /var/log | grep nifi 8621255 0 drwxr-xr-x. 2 nifinifi 91 Jan 30 17:10 nifi
[nifi@nifi1n1 run]$ cd /var/run [nifi@nifi1n1 run]$ ls -lisa | grep nifi 62714 0 drwxr-xr-x. 2 nifi hadoop 40 Jan 30 17:11 nifi if I execute export JAVA_HOME=/usr/jdk64/jdk1.8.0_60;/opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log from bash the service goes up. ... and I am getting crazy 🙂 Any Advise???
... View more
01-30-2016
03:51 PM
1 Kudo
I think I am messing up things here 🙂 ... I initially put the user nifi in the groop hadoop ... and did the nifi install ... as what it looks like ambari requires user "nifi" to be in group "nifi" ... I will start to change a couple of access rights and provide further feedback when it's done ...
... View more
01-30-2016
03:40 PM
1 Kudo
I did that already in order to trick Ambari after starting nifi through bash (and then I inserted a valid PID in /var/run/nifi) and restarted the service through Amari. Obviously Ambari found that PID and got green ... but still no luck starting the service through Ambari 😞
... View more
01-30-2016
03:05 PM
1 Kudo
Hi Neeray, thanks for your feedback ... I am now getting more detailed logs ... even though the problem didn't go aways, please see above. Thanks, Rainer
... View more
01-30-2016
03:04 PM
1 Kudo
Based on Neerjas comment below I enabled user nifi to write into /var/log/ ... no at least I am getting an error in the Ambari Console when starting the service that I attached in errors-1795.txt. In addition I posted the logs of /var/log/nifi. Again, when I am starting "nifi.sh start" from bash, nifi starts successfully. Thanks, in advance, for any further advise. br, Rainer
... View more
01-30-2016
10:53 AM
2 Kudos
Dear Experts, I installed HDF (nifi-1.1.1.0-12) using a user nifi (group hadoop) under /opt/nifi-1.1.1.0-12 Starting/Stopping the nifi service from bash works fine. Afterwards I installed the Ambari Service for Nifi as outlined on https://github.com/abajwa-hw/ambari-nifi-service Unfortunately whenever I start Nifi under Ambari it goes down without showing an error in '/var/lib/ambari-agent/data/errors-xxxx.txt' the stout of ambari looks like the following: --------------------------- stdout:
2016-01-30 10:31:01,040 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485
2016-01-30 10:31:01,040 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0
2016-01-30 10:31:01,040 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-01-30 10:31:01,061 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '')
2016-01-30 10:31:01,062 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-01-30 10:31:01,082 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0')
2016-01-30 10:31:01,082 - Ensuring that hadoop has the correct symlink structure
2016-01-30 10:31:01,083 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-01-30 10:31:01,192 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485
2016-01-30 10:31:01,192 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0
2016-01-30 10:31:01,192 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1}
2016-01-30 10:31:01,214 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '')
2016-01-30 10:31:01,214 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False}
2016-01-30 10:31:01,235 - checked_call returned (0, '/usr/hdp/2.3.4.0-3485/hadoop/conf -> /etc/hadoop/2.3.4.0-3485/0')
2016-01-30 10:31:01,235 - Ensuring that hadoop has the correct symlink structure
2016-01-30 10:31:01,235 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-01-30 10:31:01,237 - Group['hadoop'] {}
2016-01-30 10:31:01,237 - Group['nifi'] {}
2016-01-30 10:31:01,238 - Group['users'] {}
2016-01-30 10:31:01,238 - User['zookeeper'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,238 - User['ams'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,239 - User['ambari-qa'] {'gid': 'hadoop', 'groups': [u'users']}
2016-01-30 10:31:01,239 - User['kafka'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,240 - User['hdfs'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,240 - User['yarn'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,241 - User['nifi'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,242 - User['mapred'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,242 - User['hbase'] {'gid': 'hadoop', 'groups': [u'hadoop']}
2016-01-30 10:31:01,243 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-30 10:31:01,244 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2016-01-30 10:31:01,248 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2016-01-30 10:31:01,248 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase', 'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2016-01-30 10:31:01,249 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2016-01-30 10:31:01,250 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] {'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2016-01-30 10:31:01,254 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to not_if
2016-01-30 10:31:01,254 - Group['hdfs'] {'ignore_failures': False}
2016-01-30 10:31:01,254 - User['hdfs'] {'ignore_failures': False, 'groups': [u'hadoop', u'hdfs']}
2016-01-30 10:31:01,255 - Directory['/etc/hadoop'] {'mode': 0755}
2016-01-30 10:31:01,266 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2016-01-30 10:31:01,267 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 0777}
2016-01-30 10:31:01,277 - Execute[('setenforce', '0')] {'not_if': '(! which getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2016-01-30 10:31:01,285 - Skipping Execute[('setenforce', '0')] due to only_if
2016-01-30 10:31:01,285 - Directory['/var/log/hadoop'] {'owner': 'root', 'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2016-01-30 10:31:01,287 - Directory['/var/run/hadoop'] {'owner': 'root', 'group': 'root', 'recursive': True, 'cd_access': 'a'}
2016-01-30 10:31:01,288 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs', 'recursive': True, 'cd_access': 'a'}
2016-01-30 10:31:01,291 - File['/usr/hdp/current/hadoop-client/conf/commons-logging.properties'] {'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2016-01-30 10:31:01,292 - File['/usr/hdp/current/hadoop-client/conf/health_check'] {'content': Template('health_check.j2'), 'owner': 'hdfs'}
2016-01-30 10:31:01,293 - File['/usr/hdp/current/hadoop-client/conf/log4j.properties'] {'content': ..., 'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2016-01-30 10:31:01,300 - File['/usr/hdp/current/hadoop-client/conf/hadoop-metrics2.properties'] {'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2016-01-30 10:31:01,301 - File['/usr/hdp/current/hadoop-client/conf/task-log4j.properties'] {'content': StaticFile('task-log4j.properties'), 'mode': 0755}
2016-01-30 10:31:01,305 - File['/etc/hadoop/conf/topology_mappings.data'] {'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if': 'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2016-01-30 10:31:01,308 - File['/etc/hadoop/conf/topology_script.py'] {'content': StaticFile('topology_script.py'), 'only_if': 'test -d /etc/hadoop/conf', 'mode': 0755}
2016-01-30 10:31:01,488 - File['/opt/nifi-1.1.1.0-12/conf/nifi.properties'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 10:31:01,491 - File['/opt/nifi-1.1.1.0-12/conf/bootstrap.conf'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 10:31:01,495 - File['/opt/nifi-1.1.1.0-12/conf/logback.xml'] {'owner': 'nifi', 'content': InlineTemplate(...), 'group': 'nifi'}
2016-01-30 10:31:01,496 - Execute['echo pid file /var/run/nifi/nifi.pid'] {}
2016-01-30 10:31:01,498 - Execute['echo JAVA_HOME=/usr/jdk64/jdk1.8.0_60'] {}
2016-01-30 10:31:01,501 - Execute['export JAVA_HOME=/usr/jdk64/jdk1.8.0_60;/opt/nifi-1.1.1.0-12/bin/nifi.sh start >> /var/log/nifi/nifi-setup.log'] {'user': 'nifi'}
2016-01-30 10:31:04,558 - Execute['cat /opt/nifi-1.1.1.0-12/bin/nifi.pid | grep pid | sed 's/pid=\(\.*\)/\1/' > /var/run/nifi/nifi.pid'] {}
2016-01-30 10:31:04,567 - Execute['chown nifi:nifi /var/run/nifi/nifi.pid'] {}
--------------------------- Again, when I start nifi from bash it works as expacted. Any help on how to fix this or better trace the problem is highly appreciated! br, Rainer
... View more
Labels:
- Labels:
-
Apache NiFi
-
Cloudera DataFlow (CDF)