Zeppelin JDBC Interpreter: Hive Error in doAs

[#1]

I just upgraded from HDP-2.6.1 to HDP-2.6.2

Both are kerberizzed clusters and doAs=True in Hive.

In 2.6.1, jdbc hive interpreter was working fine.After upgrading, even simple queries like 'Show Databases' results in error in doAs.

My configurations are same in both the versions.

org.apache.zeppelin.interpreter.InterpreterException: Error in doAs
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:415)
at org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:633)
at org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:733)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:101)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:502)
at org.apache.zeppelin.scheduler.Job.run(Job.java:175)
at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.UndeclaredThrowableException
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1884)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:407)
... 13 more
Caused by: java.sql.SQLException: Could not open client transport for any of the Server URI's in ZooKeeper: java.net.ConnectException: Connection refused
at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:218)
at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:156)
at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:79)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:205)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.commons.dbcp2.PoolingDriver.connect(PoolingDriver.java:129)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at org.apache.zeppelin.jdbc.JDBCInterpreter.getConnectionFromPool(JDBCInterpreter.java:362)
at org.apache.zeppelin.jdbc.JDBCInterpreter.access$000(JDBCInterpreter.java:89)
at org.apache.zeppelin.jdbc.JDBCInterpreter$1.run(JDBCInterpreter.java:410)
at org.apache.zeppelin.jdbc.JDBCInterpreter$1.run(JDBCInterpreter.java:407)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
... 14 more
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:185)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:248)
at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssuming

[#2]

Have you checked ?

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_release-notes/content/known_issues.html

Description of Problem: After upgrading from Ambari 2.4.2 to Ambari 2.5.2 and subsequent HDP stack upgrade from 2.5 to 2.6, jdbc(hive) interpreter fails to work correctly in Zeppelin.

Error Message: You might see one of the following errors in the Zeppelin stacktrace after running jdbc(hive):

• Error in doAs
• Failed to validate proxy privilege of zeppelin

Workaround:

1. Make sure hadoop.proxyuser.zeppelin.groups=* and hadoop.proxyuser.zeppelin.hosts=* are set in HDFS core-site.xml. If not, then configure these properties and restart all stale services. (AMBARI-21772 is currently tracking this item).
2. Make user hive.url is configured correctly in Zeppelin's JDBC hive interpreter.

   	Note
   The URL configured might be wrong, especially on secured and/or wire-encrypted clusters, due to a known issue that we will address in a future release.

3. Restart HS2.

[#3]

@Dinesh Chitlangia

I had verified all the listed know issues mentioned in the above URL.

Still the problem exists.

[#4 · Accepted solution]

@Ramya Jayathirtha

Verify Hive Service status and try connecting to hive using beeline to ensure hive is working fine.

Then, try adding the following parameter and value in your jdbc interpreter.

hive.proxy.user.property = hive.server2.proxy.user

[#5]

Thank you @Dinesh Chitlangia.

This solved the issue.