hive metastore authentication

[#1]

I got Mysql which is metadata for oozie,hive,amabri. this was setup before cluster is setup. I do not see mysql as a service in hive home. I've enabled local MIT KDC and i see below in metastore.log...i donot see it anywhere it is being authenticated to kdc

2017-02-03 11:00:13,343 INFO [main]: timeline.HadoopTimelineMetricsSink (HadoopTimelineMetricsSink.java:init(82)) - Initializing Timeline metrics sink. 2017-02-03 11:00:13,345 INFO [main]: timeline.HadoopTimelineMetricsSink (HadoopTimelineMetricsSink.java:init(100)) - Identified hostname = master2.chrsv.com, serviceName = hivemetastore 2017-02-03 11:00:14,257 INFO [main]: timeline.HadoopTimelineMetricsSink (HadoopTimelineMetricsSink.java:init(118)) - Collector Uri: http://worker1.chrsv.com:6188/ws/v1/timeline/metrics 2017-02-03 11:00:14,592 INFO [main]: impl.MetricsSinkAdapter (MetricsSinkAdapter.java:start(206)) - Sink timeline started 2017-02-03 11:00:15,133 INFO [main]: impl.MetricsSystemImpl (MetricsSystemImpl.java:startTimer(376)) - Scheduled snapshot period at 10 second(s). 2017-02-03 11:00:15,133 INFO [main]: impl.MetricsSystemImpl (MetricsSystemImpl.java:start(192)) - hivemetastore metrics system started 2017-02-03 11:00:15,938 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:newRawStore(667)) - 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 2017-02-03 11:00:16,495 INFO [main]: metastore.ObjectStore (ObjectStore.java:initializeHelper(370)) - ObjectStore, initialize called 2017-02-03 11:00:26,552 INFO [main]: metastore.ObjectStore (ObjectStore.java:getPMF(474)) - Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,Database,Type,FieldSchema,Order" 2017-02-03 11:00:39,897 INFO [main]: metastore.MetaStoreDirectSql (MetaStoreDirectSql.java:<init>(138)) - Using direct SQL, underlying DB is MYSQL 2017-02-03 11:00:39,915 INFO [main]: metastore.ObjectStore (ObjectStore.java:setConf(284)) - Initialized ObjectStore 2017-02-03 11:00:41,013 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:createDefaultRoles_core(741)) - Added admin role in metastore 2017-02-03 11:00:41,034 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:createDefaultRoles_core(750)) - Added public role in metastore 2017-02-03 11:00:41,131 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:addAdminUsers_core(790)) - No user is added in admin role, since config is empty 2017-02-03 11:00:41,139 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:init(525)) - Begin calculating metadata count metrics. 2017-02-03 11:00:41,233 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:init(527)) - Finished metadata count metrics: 1 databases, 0 tables, 0 partitions. 2017-02-03 11:00:42,847 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(6298)) - Starting DB backed MetaStore Server with SetUGI enabled 2017-02-03 11:00:42,861 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(6352)) - Started the new metaserver on port [9083]... 2017-02-03 11:00:42,862 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(6354)) - Options.minWorkerThreads = 200 2017-02-03 11:00:42,862 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(6356)) - Options.maxWorkerThreads = 100000 2017-02-03 11:00:42,862 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(6358)) - TCP keepalive = true

[#2]

I can't speak to the logging issue just yet, but is there a problem with the cluster behavior? Can you:

kinit -k -t keytab principal

Connection string to connect with beeline

!connect jdbc:hive2://hostname:10000/default;principal=hive/_HOST@REALM

[#3]

@Ameet Paranjape

[hive@master2 ~]$ kinit -k -t /etc/security/keytabs/hive.service.keytab

kinit: Cannot determine realm for host (principal host/master2.chrsv.com@)

Not sure why it is not picking since all these were setup by Ambari...howevr when i do kadmin i can see the principle as

hive/master1.chrsv.com@KERBEROS.COM hive/master2.chrsv.com@KERBEROS.COM hive/worker1.chrsv.com@KERBEROS.COM hive/worker2.chrsv.com@KERBEROS.COM

[#4]

my bad..i didnot check syntax earlier...

[hive@master2 ~]$ kinit -k -t /etc/security/keytabs/hive.service.keytab hive/master2.chrsv.com@KERBEROS.COM [hive@master2 ~]$ klist Ticket cache: FILE:/tmp/krb5cc_501 Default principal: hive/master2.chrsv.com@KERBEROS.COM Valid starting Expires Service principal 02/03/17 14:55:41 02/04/17 14:55:41 krbtgt/KERBEROS.COM@KERBEROS.COM renew until 02/03/17 14:55:41 [hive@master2 ~]$

[#5]

Looks like the kinit is working. Did you try beeline connection and was is successful?

[#6]

@Ameet Paranjape we are talking about hive metastore....hiveserver itself is not talking to hivemetastore...so connecting to beeline does not help.

[#7 · Accepted solution]

this is because mysql is external to ambari and when kerberos is enabled ambari is not smart enough to recognize mysql and it didnot create keytabs for mysql. that was the reason hive was not able to start.

i still need to find out a way to create keytabs for non ambari components. as of now i moved these components to another server where all the services were deployed through ambari.

thanks to all for your help so far.