Created 12-05-2017 02:14 PM
I am using Zeppelin and Hive in a single Hadoop cluster (Pseudo-distributed mode). I have edited jdbc in Zeppelin interpreter and I have added hive settings as suggested in https://zeppelin.apache.org/docs/0.7.3/interpreter/hive.html
Now, I am executing the below commands in Zeppelin
%jdbc(hive) show databases;
But, I am facing with the following error:
java.sql.SQLException: Could not establish connection to jdbc:hive2://localhost:10000: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:14:13, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:336, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:279, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:189, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:414, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:310, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1377, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1362, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1152, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:622, java.lang.Thread:run:Thread.java:748, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:22:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:89, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:421, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1807, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy34:open::-1, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:327, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:29:7, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:578, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:513, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:165, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:User: root is not allowed to impersonate anonymous:54:25, org.apache.hadoop.ipc.Client:getRpcResponse:Client.java:1481, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1337, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:227, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:116, com.sun.proxy.$Proxy30:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:787, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:398, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeMethod:RetryInvocationHandler.java:163, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invoke:RetryInvocationHandler.java:155, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeOnce:RetryInvocationHandler.java:95, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:335, com.sun.proxy.$Proxy31:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:1700, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1436, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1436, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:674, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:622, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:550], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous), serverProtocolVersion:null) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:467) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:178) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:187) at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79) at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205) at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435) at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363) at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:233) at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:360) at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:378) at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:570) at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:709) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:97) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:498) at org.apache.zeppelin.scheduler.Job.run(Job.java:175) at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:473) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.thrift.protocol.TProtocolException: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:14:13, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:336, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:279, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:189, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:414, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:310, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1377, org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1362, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:286, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1152, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:622, java.lang.Thread:run:Thread.java:748, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:22:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:89, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:421, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1807, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy34:open::-1, org.apache.hive.service.cli.session.SessionManager:createSession:SessionManager.java:327, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous:29:7, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:578, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:513, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:165, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:User: root is not allowed to impersonate anonymous:54:25, org.apache.hadoop.ipc.Client:getRpcResponse:Client.java:1481, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1337, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:227, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:116, com.sun.proxy.$Proxy30:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:787, sun.reflect.NativeMethodAccessorImpl:invoke0:NativeMethodAccessorImpl.java:-2, sun.reflect.NativeMethodAccessorImpl:invoke:NativeMethodAccessorImpl.java:57, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:398, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeMethod:RetryInvocationHandler.java:163, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invoke:RetryInvocationHandler.java:155, org.apache.hadoop.io.retry.RetryInvocationHandler$Call:invokeOnce:RetryInvocationHandler.java:95, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:335, com.sun.proxy.$Proxy31:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:1700, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1436, org.apache.hadoop.hdfs.DistributedFileSystem$27:doCall:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1433, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1436, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:674, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:622, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:550], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate anonymous), serverProtocolVersion:null) at org.apache.hive.service.cli.thrift.TOpenSessionResp.validate(TOpenSessionResp.java:578) at org.apache.hive.service.cli.thrift.TOpenSessionResp$TOpenSessionRespStandardScheme.read(TOpenSessionResp.java:676) at org.apache.hive.service.cli.thrift.TOpenSessionResp$TOpenSessionRespStandardScheme.read(TOpenSessionResp.java:612) at org.apache.hive.service.cli.thrift.TOpenSessionResp.read(TOpenSessionResp.java:520) at org.apache.hive.service.cli.thrift.TCLIService$OpenSession_result$OpenSession_resultStandardScheme.read(TCLIService.java:2281) at org.apache.hive.service.cli.thrift.TCLIService$OpenSession_result$OpenSession_resultStandardScheme.read(TCLIService.java:2266) at org.apache.hive.service.cli.thrift.TCLIService$OpenSession_result.read(TCLIService.java:2213) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_OpenSession(TCLIService.java:156) at org.apache.hive.service.cli.thrift.TCLIService$Client.OpenSession(TCLIService.java:143) at org.apache.hive.jdbc.HiveConnection.openSession(HiveConnection.java:456) ... 27 more
Please guide me in this matter.
Created 12-05-2017 08:01 PM
I see the following error in the "Caused By" section of the error:
User: root is not allowed to impersonate anonymous), serverProtocolVersion:null)
Which makes me feel that your hadoop proxy user configuration is not setup correctly.
Please check the proxy user settings inside the
Ambari UI --> Services --> HDFS -- > Configs --> Advanced tab --> Custom core-site
If not added then, Click Add Property… to add the following custom properties.
hadoop.proxyuser.root.groups=* hadoop.proxyuser.root.hosts=*
.
Also please double check if the "localhost" is the correct address for your HiveServer2? Is that port listening? Can you please try changing it to proper Hostname instead of "localhost". Please run the following commands on the HS2 host to see if the hostname is correct and the port is listening.
# netstat -tnlpa | grep 10000 # hostname -f
.
Created 12-05-2017 09:53 PM
Good to know that the issue is resolved. As the issue is resolved, hence it will be also great if you can mark this HCC thread as Answered by clicking on the "Accept" Button on the correct answer. That way other HCC users can quickly find the solution when they encounter the same issue.
Created 12-05-2017 10:06 PM
Dear @Aditya Sirna and @Jay Kumar SenSharma,
Thank you for your valuable comments and help. I could resolve issues regarding to running Zeppelin HIVE into standalone Hadoop cluster. I have added following configurations to sort out the issue:
1- I have added hadoop.proxyuser.hiveuser.hosts=* and hadoop.proxyuser.hiveuser.groups=* into core-site.xml in Hadoop.
2- I copied HIVE JDBC jar file from a standalone cluster into Zeppelin JDBC interpreter: cp /usr/local/hive/lib/hive-jdbc-2.1.0.jar /usr/local/zeppelin/interpreter/jdbc/
3- I ran /usr/local/hive/bin/hiveserver2 to be able to listen port 10000.
4- I configured dependencies in jdbc interpreter from Zeppelin Web UI as follows:
org.apache.hive:hive-jdbc::2.1.0 (because I have installed Hive version 2.1.0)
org.apache.hadoop:hadoop-common::2.8.1 (because I have installed Hadoop version 2.8.1)