question Re: Getting error while opening hive shell in Support Questions

Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 16:13:03 GMT

$ hive

Logging initialized using configuration in jar:file:/chroot/hadoop/inst/hive/apache-hive-0.13.1-bin/lib/hive-common-0.13.1.jar!/hive-log4j.properties

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340) ... 7 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410) ... 12 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:185)

Please help to resolve this error.

Re: Getting error while opening hive shell

nyakkanti — Wed, 20 Apr 2016 16:18:52 GMT

@Amit Dass Can you please check Hive Metastore logs for any errors/exceptions. Assuming you are using mysql as Metadata. Further Can you check if mysql is running and you can ping/telnet to mysql with the setting in the hive-site.xml and connect mysql with username/pwd provided in hive-site.xml

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 16:27:33 GMT

Yes user name and passwords are there at <value> tag of : <name>javax.jdo.option.ConnectionUserName</name> , <name>javax.jdo.option.ConnectionPassword</name> . Can you please elloborate the steps to check whether mysql is working or not .

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 16:28:15 GMT

@nyakkanti

Re: Getting error while opening hive shell

nyakkanti — Wed, 20 Apr 2016 16:31:46 GMT

Have you checked hivemetastore.log, did you find any errors ?

Re: Getting error while opening hive shell

KuldeepK — Wed, 20 Apr 2016 17:12:17 GMT

@Amit Dass - Looks like your HiveMetastore process is not running, I can see connection refused from the logs

Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused at org.apache.thrift.transport.TSocket.open(TSocket.java:185)

Or there might be some connection issues with hive client and hive metastore machine.

Can you please check connectivity as well ?

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 17:16:48 GMT

@Kuldeep Kulkarni : Can you please brief that how should I check connectivity?

Re: Getting error while opening hive shell

KuldeepK — Wed, 20 Apr 2016 17:22:29 GMT

@Amit Dass

From hive client (from where you are running hive command) check if you are able to run below command

telnet <hive-metastore-hostname/ip-address> 9083

On metastore:

try running below command to check is metstore process is running

ps aux|grep HiveMetaStore

try running below command to ensure that metastore process is listening on 9038

root@sandbox ~]# netstat -tulpn|grep 9083
tcp        0      0 0.0.0.0:9083                0.0.0.0:*                   LISTEN      2115/java
[root@sandbox ~]#

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 17:27:58 GMT

Sorry can you please suggest where this file should be present? @nyakkanti

Re: Getting error while opening hive shell

jyadav — Wed, 20 Apr 2016 17:29:29 GMT

@Amit Dass

Can you please follow below steps and share the result?.

1. Find out the node which hosting hive metastore service, you can use ambari to figure out the ip/hostname for that node.

2. ssh root login to that hive metastore node and execute below commands.

bash# ps -aef|grep -i org.apache.hadoop.hive.metastore.HiveMetaStore

bash# lsof -i:9083

3. From some other node try connecting hivemetastore port using telnet.

bash# telnet <hivemeta server> 9083

If 9083 port is not occupied and ps command doesn't show any metastore process then please try to restart the hivemestore service from ambari UI and perform same check again.

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 17:34:01 GMT

@Kuldeep Kulkarni : Getting connection refused as below also why we use port 9038

telnet xx.xx.xx.xx 9038

Trying xx.xx.xx.xx... telnet: connect to address xx.xx.xx.xx: Connection refused

Re: Getting error while opening hive shell

KuldeepK — Wed, 20 Apr 2016 17:45:20 GMT

Sorry I have corrected port number, please try the steps given above and let me know how it goes

Re: Getting error while opening hive shell

KuldeepK — Wed, 20 Apr 2016 17:45:32 GMT

@Amit Dass

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 17:55:03 GMT

@Jitendra Yadav : Thanks , I am using distributed system which has one namenode and two datanode . This for POC purpose only . Can you please elaborate more on this .

I logged in Namenode assume : xx.xx.xx.xx

Hadoop hdfs commands working fine.

But when I try to enter into hive shell it throwing error.

1. As hive installed in Namenode so we had IP of namenode.

2. ps -aef|grep -i org.apache.hadoop.hive.metastore.HiveMetaStore

o/p : hadoop 15097 13478 0 16:21 pts/2 00:00:00 grep -i org.apache.hadoop.hive.metastore.HiveMetaStore

lsof -i:9083

o/p : Did not get any output for the same

3. Getting connection refused .. telnet < Namenode IP> 9083

This is simple unix machine which had hadoop , hive ,hbase installed. Please guide me on this.

Re: Getting error while opening hive shell

KuldeepK — Wed, 20 Apr 2016 18:42:47 GMT

@Amit Dass - I'm assuming you have installed cluster with Ambari, Please go to the ambari server UI and check on which host hive metostore is installed, you can go to the hive services and click on metastore service to check the same.

If Ambari shows metastore as down then you need to start it.

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 18:47:59 GMT

@Kuldeep Kulkarni :No I am not using Ambari

I am using distributed system which has one namenode and two datanode . This for POC purpose only .

This is simple unix machine which had hadoop , hive ,hbase installed. Please guide me accordingly.

Re: Getting error while opening hive shell

jyadav — Wed, 20 Apr 2016 20:33:46 GMT

@Amit Dass

So looks like your hive metastore service is not running, Would you please try to start metastore service from ambari UI? please see attachement.screen-shot-2016-04-20-at-23103-pm.png

Re: Getting error while opening hive shell

amit_dass — Wed, 20 Apr 2016 20:49:58 GMT

@Jitendra Yadav : Hi , I didnot have Ambari installed . Do we have any command which I can run to start metastore service.

Re: Getting error while opening hive shell

jyadav — Wed, 20 Apr 2016 21:28:43 GMT

@Amit Dass

Please below command to start the hivemetastore..

su $HIVE_USER 
nohup /usr/hdp/current/hive-metastore/bin/hive --service metastore>/var/log/hive/hive.out 2>/var/log/hive/hive.log &

Re: Getting error while opening hive shell

amit_dass — Thu, 21 Apr 2016 13:24:40 GMT

Thanks @Jitendra Yadav . Hive --service metastore now I can access hive shell .

But ... only when I start metastore service in another terminal . As I close this metastore window .. Hive stop and throw exception

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTa sk. MetaException(message:Got exception: org.apache.thrift.transport.TTransportE xception null)