Reply
New Contributor
Posts: 4
Registered: ‎04-16-2014

Issues of Hive metastore connection by hive client in Kerberos envrionment on CDH5.

[ Edited ]

When I enabled Kerberos on CDH5,and integrated kerberos with all services on CDH5,I met some problems with hive query action by Webhcat.The other services ,such as mapreduce pig which don't use hive metastore can work.

The following is the issue log of  hive metastore server:

 

2014-04-24 13:23:54,364 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: 4: get_databases: NonExistentDatabaseUsedForHealthCheck
2014-04-24 13:23:54,365 INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=HTTP/cdh78.xxx.com@EXAMPLE.COM ip=/192.168.73.78 cmd=get_databases: NonExistentDatabaseUsedForHe
althCheck
2014-04-24 13:23:54,365 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: 4: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2014-04-24 13:23:54,368 INFO org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize called
2014-04-24 13:23:54,371 INFO org.apache.hadoop.hive.metastore.MetaStoreDirectSql: MySQL check failed, assuming we are not on mysql: ERROR: syntax error at or near "@@"
Position: 5
2014-04-24 13:23:54,375 INFO DataNucleus.Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing
2014-04-24 13:23:54,376 INFO org.apache.hadoop.hive.metastore.ObjectStore: Initialized ObjectStore
2014-04-24 13:23:54,381 WARN org.apache.hadoop.hive.metastore.ObjectStore: Metastore version was 0.12.0 hive.metastore.schema.verification is not enabled so recording the new schema version 0
.12.0
2014-04-24 13:23:54,392 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: 4: cancel_delegation_token
2014-04-24 13:23:54,392 INFO org.apache.hadoop.hive.metastore.HiveMetaStore.audit: ugi=HTTP/cdh78.xxx.com@EXAMPLE.COM ip=/192.168.73.78 cmd=cancel_delegation_token
2014-04-24 13:23:54,392 INFO org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager: Token cancelation requested for identifier: owner=HTTP, renewer=HTTP, realUser=, issueDate=1
398316916994, maxDate=1398921716994, sequenceNumber=2, masterKeyId=2
2014-04-24 13:31:30,549 ERROR org.apache.thrift.server.TThreadPoolServer: Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:654)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:652)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:356)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1528)
at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:652)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:227)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
at org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:182)
at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)

 

please give me help.

 

 

Cloudera Employee
Posts: 508
Registered: ‎07-30-2013

Re: Issues of Hive metastore connection by hive client in Kerberos envrionment on CDH5.

Hi Steven,

 

WebHCat is only supported for metastore queries, not query execution. You should use HiveServer2  or Oozie to submit hive jobs.

 

Thanks,

Darren

Announcements