Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Error while running hive queries from Zeppelin on HDP 2.3.2

avatar

Am getting error "Required field 'serverProtocolVersion' is unset" when trying to run Hive queries from recent build of Zeppelin on HDP 2.3.2/Spark 1.4.1. Tried both Zeppelin service and compiling Zeppelin. Spark and Phoenix jobs run fine. Hive used to work fine before.

Google search suggests issue may be caused by below...anyone else run into this?

https://issues.apache.org/jira/browse/HIVE-6050

http://stackoverflow.com/questions/24694415/required-field-client-protocol-is-unset

Using default proxy settings for Hive

hadoop.proxyuser.hive.hosts = *

hadoop.proxyuser.hive.groups = users

Full error

ERROR [2015-11-27 04:41:26,630] ({pool-2-thread-2} HiveConnection.java[openSession]:466) - Error opening session

org.apache.thrift.protocol.TProtocolException: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive:13:12, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:266, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:202, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:402, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:297, org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1253, org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1238, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615, java.lang.Thread:run:Thread.java:745, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive:21:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:83, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:415, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1657, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy19:open::-1, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:258, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive:26:5, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:522, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:137, sun.reflect.GeneratedMethodAccessor10:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:User: hive is not allowed to impersonate hive:45:19, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1358, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:229, com.sun.proxy.$Proxy14:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:771, sun.reflect.GeneratedMethodAccessor7:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:187, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:102, com.sun.proxy.$Proxy15:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:2116, org.apache.hadoop.hdfs.DistributedFileSystem$22:doCall:DistributedFileSystem.java:1305, org.apache.hadoop.hdfs.DistributedFileSystem$22:doCall:DistributedFileSystem.java:1301, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1301, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1424, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:596, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:554, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:508], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive), serverProtocolVersion:null)
1 ACCEPTED SOLUTION

avatar

(Adding as answer as it too big to add as comment): Ran some tests and here are the findings. Changing proxyuser helps but still having issues getting queries that result in Tez jobs working for some reason. With 2.3.0 sandbox (spark 1.3.1) all hive queries seem to run fine

Test 1:

  • 2.3.0.0-2557 (sandbox)
  • HDP Spark 1.3.1
  • hadoop.proxyuser.hive.groups=users
  • Results: all hive queries works fine

Test 2:

  • 2.3.2.0-2950
  • HDP Spark 1.4.1
  • hadoop.proxyuser.hive.groups=users
  • Results: Error when running any hive query: Required field 'serverProtocolVersion' is unset
  • running from hive cli as zeppelin user runs fine

Test 3:

  • Ambari installed 2.3.2.0-2950 env
  • HDP Spark 1.4.1
  • hadoop.proxyuser.hive.groups=*
  • hive queries that don’t generate tez job work fine
  • Error when running hive query that generate tez job : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
  • running from hive cli as zeppelin user runs fine

Test 4:

  • Ambari installed 2.3.2.0-2950 env
  • Installed Spark 1.5.1 TP
  • hadoop.proxyuser.hive.groups=users
  • Results: Error when running any hive query: Required field 'serverProtocolVersion' is unset
  • running from hive cli as zeppelin user runs fine

Test 5:

  • Ambari installed 2.3.2.0-2950 env
  • Installed Spark 1.5.1 TP
  • hadoop.proxyuser.hive.groups=*
  • hive queries that don’t generate tez job work fine
  • Error when running hive query that generate tez job : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
  • running from hive cli as zeppelin user runs fine

View solution in original post

10 REPLIES 10

avatar

@Ali Bajwa

I got this error when trying to connect to spark 1.4.1 thrift server using beeline that comes with sandbox 2.3.2. Spark 1.5.2 thrift server works fine with beeline 2.3.2.

Looks like new zeppelin is using new sql client, what spark version are you using?

avatar

Thanks @Guilherme Braccialli yes this is with HDP 2.3.2 / Spark 1.4.1. Interesting...so using Spark 1.5.2 will fix this? I will try this out.

avatar

It should. Actually I'm using Spark 1.5.1 Technical Preview, but I think 1.5.2 should work also.

avatar

@Ali Bajwa saw the same error with spark 1.4.1, it did work with 1.5.1, so it looks like its fixed.

avatar
Master Mentor

@Ali Bajwa Just curious ..what happens if you set

hadoop.proxyuser.hive.groups = *

avatar

(Adding as answer as it too big to add as comment): Ran some tests and here are the findings. Changing proxyuser helps but still having issues getting queries that result in Tez jobs working for some reason. With 2.3.0 sandbox (spark 1.3.1) all hive queries seem to run fine

Test 1:

  • 2.3.0.0-2557 (sandbox)
  • HDP Spark 1.3.1
  • hadoop.proxyuser.hive.groups=users
  • Results: all hive queries works fine

Test 2:

  • 2.3.2.0-2950
  • HDP Spark 1.4.1
  • hadoop.proxyuser.hive.groups=users
  • Results: Error when running any hive query: Required field 'serverProtocolVersion' is unset
  • running from hive cli as zeppelin user runs fine

Test 3:

  • Ambari installed 2.3.2.0-2950 env
  • HDP Spark 1.4.1
  • hadoop.proxyuser.hive.groups=*
  • hive queries that don’t generate tez job work fine
  • Error when running hive query that generate tez job : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
  • running from hive cli as zeppelin user runs fine

Test 4:

  • Ambari installed 2.3.2.0-2950 env
  • Installed Spark 1.5.1 TP
  • hadoop.proxyuser.hive.groups=users
  • Results: Error when running any hive query: Required field 'serverProtocolVersion' is unset
  • running from hive cli as zeppelin user runs fine

Test 5:

  • Ambari installed 2.3.2.0-2950 env
  • Installed Spark 1.5.1 TP
  • hadoop.proxyuser.hive.groups=*
  • hive queries that don’t generate tez job work fine
  • Error when running hive query that generate tez job : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
  • running from hive cli as zeppelin user runs fine

avatar
Master Mentor

avatar

nice one!

avatar
Guru

hadoop.proxyuser.hive.groups = *

Worked for me. Thanks @Ali Bajwa