Welcome to the Cloudera Community

Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Who agreed with this topic

hive.metastore.client.impl and hive.sentry.conf.url in CDH 5.4

avatar
Explorer

Hi all,

 

I ran into this weird problem and not sure how to fix this.  I started metastore and server2 and in both /var/log/hive for these services showed similar WARN messages:

 


15/06/01 15:02:37 WARN conf.HiveConf: HiveConf of name hive.metastore.client.impl does not exist
15/06/01 15:02:37 WARN conf.HiveConf: HiveConf of name hive.sentry.conf.url does not exist

 

But there are entries for these in /etc/hive/conf/hive-site.xml
  <property>
    <name>hive.sentry.conf.url</name>
    <value>file:///etc/sentry/conf/sentry-site.xml</value>
  </property
...
  <property>
    <name>hive.metastore.client.impl</name>
    <value>org.apache.sentry.binding.metastore.SentryHiveMetaStoreClient</value>
    <description>Sets custom Hive Metastore client which Sentry uses to filter out metadata.</description>
  </property>

And /etc/sentry/conf/sentry-site.xml does exist and readable by hive user that runs hive metastore and server2 on the same server.
[ec2-user@cdh54-kerldap-hive hive]$ ls -lhat /etc/sentry/conf/sentry-site.xml
-rw-r--r-- 1 hive hive 865 Jun  1 10:44 /etc/sentry/conf/sentry-site.xml
 
 
I certainly want to use Sentry and here is my sentry-site.xml
 
<configuration>
  <property>
    <name>hive.sentry.provider</name>
    <value>org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider</value>
<!--    <value>org.apache.sentry.provider.file.LocalGroupResourceAuthorizationProvider</value> -->

  </property>

  <property>
    <name>hive.sentry.provider.resource</name>
    <value>hdfs://cdh54kerldaphdfs/user/secconfig/sentry/sentry-provider.ini</value>
    <!--
       If the hdfs-site.xml points to HDFS, the path will be in HDFS;
       alternatively you could specify a full path, e.g.:
       hdfs://namenode:port/path/to/authz-provider.ini
       file:///path/to/authz-provider.ini
    -->
  </property>

  <property>
      <name>sentry.hive.server</name>
      <description>must be same on Impala and Hive</description>
      <value>server1</value>
  </property>
</configuration>
 
In this setup, I have a KDC + openldap backend + HA NNs/JTs and when I used beeline to connect, it seems to work initially but I have a feeling that this does  not look right as
  <property>
    <name>hive.sentry.conf.url</name>
    <value>file:///etc/sentry/conf/sentry-site.xml</value>
  </property
 
could not be found => most likely sentry is not going to work.
 
 
    [root@cdh54-kerldap-hive hive]# kinit mydomain
    Password for user1@MYDOMAINDEV.COM:
    [root@cdh54-kerldap-hive hive]# beeline --verbose
    Beeline version 1.1.0-cdh5.4.0 by Apache Hive
    beeline> !connect jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM
    scan complete in 2ms
    Connecting to jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM
    Enter username for jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM: hiveuser
    Enter password for jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM: ********
    Connected to: Apache Hive (version 1.1.0-cdh5.4.0)
    Driver: Hive JDBC (version 1.1.0-cdh5.4.0)
    Transaction isolation: TRANSACTION_REPEATABLE_READ
    0: jdbc:hive2://localhost:10000/default> show tables;
    Getting log thread is interrupted, since query is done!                     
    +-----------+--+
    | tab_name  |
    +-----------+--+
    +-----------+--+
    No rows selected (3.683 seconds)
 
I turned logging to DEBUG level and here are what I found in hive-metastore.out. Here is the pastebin to this file http://pastebin.com/jZMcJmyJ
Here is the pastebin to hive-server2.out http://pastebin.com/1YbaNjz4.
 
From the log of hive-server2.out, it looks like it actually work because of these lines:
 
15/06/01 15:24:04 INFO hive.metastore: Mestastore configuration hive.security.authorization.manager changed from org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider to org.apache.sentry.binding.hive.HiveAuthzBindingSessionHook$SentryHiveAuthorizerFactory
....
15/06/01 15:24:04 INFO file.SimpleFileProviderBackend: Parsing hdfs://cdh54kerldaphdfs/user/secconfig/sentry/sentry-provider.ini
15/06/01 15:24:04 INFO file.SimpleFileProviderBackend: Filesystem: hdfs://cdh54kerldaphdfs
15/06/01 15:24:04 INFO file.PolicyFiles: Opening hdfs://cdh54kerldaphdfs/user/secconfig/sentry/sentry-provider.ini
 
Please shed some lights on why there are WARN messages for these two properties.
 
Thank you very much for all your helps.
Steven.
 
 
Who agreed with this topic