Error while running hive queries from Zeppelin on HDP 2.3.2

[#1]

Am getting error "Required field 'serverProtocolVersion' is unset" when trying to run Hive queries from recent build of Zeppelin on HDP 2.3.2/Spark 1.4.1. Tried both Zeppelin service and compiling Zeppelin. Spark and Phoenix jobs run fine. Hive used to work fine before.

Google search suggests issue may be caused by below...anyone else run into this?

https://issues.apache.org/jira/browse/HIVE-6050

http://stackoverflow.com/questions/24694415/required-field-client-protocol-is-unset

Using default proxy settings for Hive

hadoop.proxyuser.hive.hosts = *

hadoop.proxyuser.hive.groups = users

Full error

ERROR [2015-11-27 04:41:26,630] ({pool-2-thread-2} HiveConnection.java[openSession]:466) - Error opening session

org.apache.thrift.protocol.TProtocolException: Required field 'serverProtocolVersion' is unset! Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS, infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive:13:12, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:266, org.apache.hive.service.cli.CLIService:openSessionWithImpersonation:CLIService.java:202, org.apache.hive.service.cli.thrift.ThriftCLIService:getSessionHandle:ThriftCLIService.java:402, org.apache.hive.service.cli.thrift.ThriftCLIService:OpenSession:ThriftCLIService.java:297, org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1253, org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession:getResult:TCLIService.java:1238, org.apache.thrift.ProcessFunction:process:ProcessFunction.java:39, org.apache.thrift.TBaseProcessor:process:TBaseProcessor.java:39, org.apache.hive.service.auth.TSetIpAddressProcessor:process:TSetIpAddressProcessor.java:56, org.apache.thrift.server.TThreadPoolServer$WorkerProcess:run:TThreadPoolServer.java:285, java.util.concurrent.ThreadPoolExecutor:runWorker:ThreadPoolExecutor.java:1145, java.util.concurrent.ThreadPoolExecutor$Worker:run:ThreadPoolExecutor.java:615, java.lang.Thread:run:Thread.java:745, *java.lang.RuntimeException:java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive:21:8, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:83, org.apache.hive.service.cli.session.HiveSessionProxy:access$000:HiveSessionProxy.java:36, org.apache.hive.service.cli.session.HiveSessionProxy$1:run:HiveSessionProxy.java:63, java.security.AccessController:doPrivileged:AccessController.java:-2, javax.security.auth.Subject:doAs:Subject.java:415, org.apache.hadoop.security.UserGroupInformation:doAs:UserGroupInformation.java:1657, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:59, com.sun.proxy.$Proxy19:open::-1, org.apache.hive.service.cli.session.SessionManager:openSession:SessionManager.java:258, *java.lang.RuntimeException:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive:26:5, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:522, org.apache.hive.service.cli.session.HiveSessionImpl:open:HiveSessionImpl.java:137, sun.reflect.GeneratedMethodAccessor10:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hive.service.cli.session.HiveSessionProxy:invoke:HiveSessionProxy.java:78, *org.apache.hadoop.ipc.RemoteException:User: hive is not allowed to impersonate hive:45:19, org.apache.hadoop.ipc.Client:call:Client.java:1427, org.apache.hadoop.ipc.Client:call:Client.java:1358, org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker:invoke:ProtobufRpcEngine.java:229, com.sun.proxy.$Proxy14:getFileInfo::-1, org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB:getFileInfo:ClientNamenodeProtocolTranslatorPB.java:771, sun.reflect.GeneratedMethodAccessor7:invoke::-1, sun.reflect.DelegatingMethodAccessorImpl:invoke:DelegatingMethodAccessorImpl.java:43, java.lang.reflect.Method:invoke:Method.java:606, org.apache.hadoop.io.retry.RetryInvocationHandler:invokeMethod:RetryInvocationHandler.java:187, org.apache.hadoop.io.retry.RetryInvocationHandler:invoke:RetryInvocationHandler.java:102, com.sun.proxy.$Proxy15:getFileInfo::-1, org.apache.hadoop.hdfs.DFSClient:getFileInfo:DFSClient.java:2116, org.apache.hadoop.hdfs.DistributedFileSystem$22:doCall:DistributedFileSystem.java:1305, org.apache.hadoop.hdfs.DistributedFileSystem$22:doCall:DistributedFileSystem.java:1301, org.apache.hadoop.fs.FileSystemLinkResolver:resolve:FileSystemLinkResolver.java:81, org.apache.hadoop.hdfs.DistributedFileSystem:getFileStatus:DistributedFileSystem.java:1301, org.apache.hadoop.fs.FileSystem:exists:FileSystem.java:1424, org.apache.hadoop.hive.ql.session.SessionState:createRootHDFSDir:SessionState.java:596, org.apache.hadoop.hive.ql.session.SessionState:createSessionDirs:SessionState.java:554, org.apache.hadoop.hive.ql.session.SessionState:start:SessionState.java:508], errorCode:0, errorMessage:Failed to open new session: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive), serverProtocolVersion:null)

[#2]

@Ali Bajwa

I got this error when trying to connect to spark 1.4.1 thrift server using beeline that comes with sandbox 2.3.2. Spark 1.5.2 thrift server works fine with beeline 2.3.2.

Looks like new zeppelin is using new sql client, what spark version are you using?

[#3]

Thanks @Guilherme Braccialli yes this is with HDP 2.3.2 / Spark 1.4.1. Interesting...so using Spark 1.5.2 will fix this? I will try this out.

[#4]

It should. Actually I'm using Spark 1.5.1 Technical Preview, but I think 1.5.2 should work also.

[#5]

@Ali Bajwa saw the same error with spark 1.4.1, it did work with 1.5.1, so it looks like its fixed.

[#6]

@Ali Bajwa Just curious ..what happens if you set

hadoop.proxyuser.hive.groups = *

[#7 · Accepted solution]

(Adding as answer as it too big to add as comment): Ran some tests and here are the findings. Changing proxyuser helps but still having issues getting queries that result in Tez jobs working for some reason. With 2.3.0 sandbox (spark 1.3.1) all hive queries seem to run fine

Test 1:

• 2.3.0.0-2557 (sandbox)
• HDP Spark 1.3.1
• hadoop.proxyuser.hive.groups=users
• Results: all hive queries works fine

Test 2:

• 2.3.2.0-2950
• HDP Spark 1.4.1
• hadoop.proxyuser.hive.groups=users
• Results: Error when running any hive query: Required field 'serverProtocolVersion' is unset
• running from hive cli as zeppelin user runs fine

Test 3:

• Ambari installed 2.3.2.0-2950 env
• HDP Spark 1.4.1
• hadoop.proxyuser.hive.groups=*
• hive queries that don’t generate tez job work fine
• Error when running hive query that generate tez job : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
• running from hive cli as zeppelin user runs fine

Test 4:

• Ambari installed 2.3.2.0-2950 env
• Installed Spark 1.5.1 TP
• hadoop.proxyuser.hive.groups=users
• Results: Error when running any hive query: Required field 'serverProtocolVersion' is unset
• running from hive cli as zeppelin user runs fine

Test 5:

• Ambari installed 2.3.2.0-2950 env
• Installed Spark 1.5.1 TP
• hadoop.proxyuser.hive.groups=*
• hive queries that don’t generate tez job work fine
• Error when running hive query that generate tez job : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask
• running from hive cli as zeppelin user runs fine

[#8]

@vshukla

[#9]

nice one!

[#10]

hadoop.proxyuser.hive.groups = *

Worked for me. Thanks @Ali Bajwa