Support Questions
Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Advanced Search

Hive MetaStore Delegation Token Error

Highlighted

Hive MetaStore Delegation Token Error

Labels:
goldjay1231
New Contributor

Created ‎07-26-2016 09:59 AM

Hi,

When I configure below in hive-site.xml, it will lead Hive MetaStore can't start.

hive.cluster.delegation.token.store.class = org.apache.hadoop.hive.thrift.DBTokenStore

But when I change to below, Hive MetaStore can start successfully.

hive.cluster.delegation.token.store.class = org.apache.hadoop.hive.thrift.MemoryTokenStore

-----------------------------------------

My environment are below:

OS: CentOS 6.6

Ambari: 2.2.2

HDP: 2.4.2

Hive: 1.2.1

Kerberos: enabled

-----------------------------------------

Hive MetaStore Log:

2016-07-25 16:45:24,505 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(5917)) - Starting hive metastore on port 9083
2016-07-25 16:45:24,736 INFO  [org.apache.hadoop.util.JvmPauseMonitor$Monitor@44fb27d8]: util.JvmPauseMonitor (JvmPauseMonitor.java:run(179)) - Starting JVM pause monitor
2016-07-25 16:45:24,807 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(599)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2016-07-25 16:45:24,890 INFO  [main]: metastore.ObjectStore (ObjectStore.java:initialize(294)) - ObjectStore, initialize called
2016-07-25 16:45:27,685 INFO  [main]: metastore.ObjectStore (ObjectStore.java:getPMF(379)) - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,Database,Type,FieldSchema,Order"
2016-07-25 16:45:30,584 INFO  [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(140)) - Using direct SQL, underlying DB is MYSQL
2016-07-25 16:45:30,592 INFO  [main]: metastore.ObjectStore (ObjectStore.java:setConf(277)) - Initialized ObjectStore
2016-07-25 16:45:31,360 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:createDefaultRoles_core(673)) - Added admin role in metastore
2016-07-25 16:45:31,369 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:createDefaultRoles_core(682)) - Added public role in metastore
2016-07-25 16:45:31,562 INFO  [main]: metastore.HiveMetaStore (HiveMetaStore.java:addAdminUsers_core(722)) - No user is added in admin role, since config is empty
2016-07-25 16:45:32,353 INFO  [main]: security.UserGroupInformation (UserGroupInformation.java:loginUserFromKeytab(979)) - Login successful for user hive/persp-3.persp.net@PERSP.NET using keytab file /etc/security/keytabs/hive.service.keytab
2016-07-25 16:45:32,375 INFO  [main]: delegation.AbstractDelegationTokenSecretManager (AbstractDelegationTokenSecretManager.java:updateCurrentKey(325)) - Updating the current master key for generating delegation tokens
2016-07-25 16:45:32,398 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(6080)) - java.io.IOException: Failed to initialize master key
    at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.startThreads(TokenStoreDelegationTokenSecretManager.java:213)
    at org.apache.hadoop.hive.thrift.HiveDelegationTokenManager.startDelegationTokenSecretManager(HiveDelegationTokenManager.java:96)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6031)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:5945)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.startThreads(TokenStoreDelegationTokenSecretManager.java:211)
    ... 9 more
Caused by: org.apache.hadoop.hive.thrift.DelegationTokenStore$TokenStoreException: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.addMasterKey(java.lang.String)
    at org.apache.hadoop.hive.thrift.DBTokenStore.invokeOnRawStore(DBTokenStore.java:156)
    at org.apache.hadoop.hive.thrift.DBTokenStore.addMasterKey(DBTokenStore.java:42)
    at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.logUpdateMasterKey(TokenStoreDelegationTokenSecretManager.java:193)
    at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.updateCurrentKey(AbstractDelegationTokenSecretManager.java:335)
    ... 14 more
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.addMasterKey(java.lang.String)
    at java.lang.Class.getMethod(Class.java:1786)
    at org.apache.hadoop.hive.thrift.DBTokenStore.invokeOnRawStore(DBTokenStore.java:146)
    ... 17 more
2016-07-25 16:45:32,399 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(5949)) - Metastore Thrift Server threw an exception...
java.io.IOException: Failed to initialize master key
    at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.startThreads(TokenStoreDelegationTokenSecretManager.java:213)
    at org.apache.hadoop.hive.thrift.HiveDelegationTokenManager.startDelegationTokenSecretManager(HiveDelegationTokenManager.java:96)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:6031)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:5945)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.startThreads(TokenStoreDelegationTokenSecretManager.java:211)
    ... 9 more
Caused by: org.apache.hadoop.hive.thrift.DelegationTokenStore$TokenStoreException: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.addMasterKey(java.lang.String)
    at org.apache.hadoop.hive.thrift.DBTokenStore.invokeOnRawStore(DBTokenStore.java:156)
    at org.apache.hadoop.hive.thrift.DBTokenStore.addMasterKey(DBTokenStore.java:42)
    at org.apache.hadoop.hive.thrift.TokenStoreDelegationTokenSecretManager.logUpdateMasterKey(TokenStoreDelegationTokenSecretManager.java:193)
    at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.updateCurrentKey(AbstractDelegationTokenSecretManager.java:335)
    ... 14 more
Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.addMasterKey(java.lang.String)
    at java.lang.Class.getMethod(Class.java:1786)
    at org.apache.hadoop.hive.thrift.DBTokenStore.invokeOnRawStore(DBTokenStore.java:146)
    ... 17 more
2016-07-25 16:45:32,405 INFO  [Thread-4]: metastore.HiveMetaStore (HiveMetaStore.java:run(5934)) - Shutting down hive metastore.

Thanks for any suggestion?

Reply
1,723 Views
0 Kudos
Tags (3)
Don't have an account?
Register