Reply
Highlighted
New Contributor
Posts: 1
Registered: ‎04-02-2014

How to authenticate to Hive CDH 4.4 using Spring-Hadoop?

[ Edited ]

How do I authenticate to Hive CDH 4.4 using Spring Hadoop?

 

I am using Spring 4.0.3.RELEASE with spring-data-hadoop 1.0.2.RELEASE-cdh4.  My Java application is running on an IaaS RHEL6 server that is configured as a Cloudera Gateway.  On the IaaS server, I can access Hive via the CLI and run queries via all three users: (1) cloud-user, (2) USER_ONE, and (3) USER-ONE.  However, when I try to use either of these methods to connect, I get a "java.net.ConnectException: Connection refused" error for every user.

 

  • Spring's HiveTemplate (host=localhost, port=10000)
  • HiveDriver + JDBC (uri=jdbc:hive://)

Note is overall Hive access is restricted to "embedded" on gateway nodes--i.e., no remote JDBC connectivity is permitted.  Security auth:SIMPLE method is currently employed. 

 

Full error stack trace:

 

2014-04-02 09:17:19,099 [main] ERROR c.e.n.r.h.HiveConsumer init() - hiveTemplate could not SHOW DATABASES: Cannot create HiveClient instance; nested exception is org.springframework.dao.DataAccessResourceFailureException: java.net.ConnectException: Connection refused; nested exception is org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.6.0_30]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327) ~[na:1.6.0_30]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193) ~[na:1.6.0_30]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180) ~[na:1.6.0_30]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:385) ~[na:1.6.0_30]
at java.net.Socket.connect(Socket.java:546) ~[na:1.6.0_30]
at org.apache.thrift.transport.TSocket.open(TSocket.java:180) ~[hive-exec-0.10.0-cdh4.4.0.jar:0.10.0-cdh4.4.0]
... 54 common frames omitted
Wrapped by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:185) ~[hive-exec-0.10.0-cdh4.4.0.jar:0.10.0-cdh4.4.0]
at org.springframework.data.hadoop.hive.HiveClientFactoryBean.createHiveClient(HiveClientFactoryBean.java:80) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
... 53 common frames omitted
Wrapped by: org.springframework.dao.DataAccessResourceFailureException: java.net.ConnectException: Connection refused; nested exception is org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.springframework.data.hadoop.hive.HiveUtils.convert(HiveUtils.java:69) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.data.hadoop.hive.HiveClientFactoryBean.createHiveClient(HiveClientFactoryBean.java:86) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.data.hadoop.hive.HiveClientFactoryBean$DefaultHiveClientFactory.getHiveClient(HiveClientFactoryBean.java:56) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
... 52 common frames omitted
Wrapped by: org.springframework.beans.factory.BeanCreationException: Cannot create HiveClient instance; nested exception is org.springframework.dao.DataAccessResourceFailureException: java.net.ConnectException: Connection refused; nested exception is org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.springframework.data.hadoop.hive.HiveClientFactoryBean$DefaultHiveClientFactory.getHiveClient(HiveClientFactoryBean.java:58) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_30]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.6.0_30]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.6.0_30]
at java.lang.reflect.Method.invoke(Method.java:622) ~[na:1.6.0_30]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at com.sun.proxy.$Proxy29.getHiveClient(Unknown Source) ~[na:na]
at org.springframework.data.hadoop.hive.HiveTemplate.createHiveClient(HiveTemplate.java:270) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.data.hadoop.hive.HiveTemplate.execute(HiveTemplate.java:78) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.data.hadoop.hive.HiveTemplate.executeScript(HiveTemplate.java:261) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.data.hadoop.hive.HiveExecutor.executeHiveScripts(HiveExecutor.java:53) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.data.hadoop.hive.HiveTasklet.execute(HiveTasklet.java:32) ~[spring-data-hadoop-1.0.2.RELEASE-cdh4.jar:1.0.2.RELEASE-cdh4]
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:404) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133) ~[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:268) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:77) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:368) ~[spring-batch-infrastructure-3.0.0.M3.jar:na]
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:215) ~[spring-batch-infrastructure-3.0.0.M3.jar:na]
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:144) ~[spring-batch-infrastructure-3.0.0.M3.jar:na]
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:254) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:198) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:144) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:64) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:67) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:163) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:142) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:134) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:304) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:135) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:50) ~[spring-core-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:128) ~[spring-batch-core-3.0.0.M3.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.6.0_30]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.6.0_30]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.6.0_30]
at java.lang.reflect.Method.invoke(Method.java:622) ~[na:1.6.0_30]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:52) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) ~[spring-aop-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at com.sun.proxy.$Proxy28.run(Unknown Source) ~[na:na]
at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:362) ~[spring-batch-core-3.0.0.M3.jar:na]
at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:588) ~[spring-batch-core-3.0.0.M3.jar:na]

 

Posts: 1,903
Kudos: 435
Solutions: 307
Registered: ‎07-31-2013

Re: How to authenticate to Hive CDH 4.4 using Spring-Hadoop?

Your Hive CLI works cause it does not talk to a JDBC service (per your URL, a HiveServer1).

Your description shows you are attempting to configure a connection to localhost:10000 of type jdbc:hive://.

The error of a ECONNREFUSED indicates no such service is running on the 'localhost' (host of app-execution).

Perhaps you will need to run a local HiveServer1 first, via command 'hive --service hiveserver 2>&1 > /tmp/hiveserver.log &' and then run your application.