Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

hive.metastore.client.impl and hive.sentry.conf.url in CDH 5.4

hive.metastore.client.impl and hive.sentry.conf.url in CDH 5.4

Explorer

Hi all,

 

I ran into this weird problem and not sure how to fix this.  I started metastore and server2 and in both /var/log/hive for these services showed similar WARN messages:

 


15/06/01 15:02:37 WARN conf.HiveConf: HiveConf of name hive.metastore.client.impl does not exist
15/06/01 15:02:37 WARN conf.HiveConf: HiveConf of name hive.sentry.conf.url does not exist

 

But there are entries for these in /etc/hive/conf/hive-site.xml
  <property>
    <name>hive.sentry.conf.url</name>
    <value>file:///etc/sentry/conf/sentry-site.xml</value>
  </property
...
  <property>
    <name>hive.metastore.client.impl</name>
    <value>org.apache.sentry.binding.metastore.SentryHiveMetaStoreClient</value>
    <description>Sets custom Hive Metastore client which Sentry uses to filter out metadata.</description>
  </property>

And /etc/sentry/conf/sentry-site.xml does exist and readable by hive user that runs hive metastore and server2 on the same server.
[ec2-user@cdh54-kerldap-hive hive]$ ls -lhat /etc/sentry/conf/sentry-site.xml
-rw-r--r-- 1 hive hive 865 Jun  1 10:44 /etc/sentry/conf/sentry-site.xml
 
 
I certainly want to use Sentry and here is my sentry-site.xml
 
<configuration>
  <property>
    <name>hive.sentry.provider</name>
    <value>org.apache.sentry.provider.file.HadoopGroupResourceAuthorizationProvider</value>
<!--    <value>org.apache.sentry.provider.file.LocalGroupResourceAuthorizationProvider</value> -->

  </property>

  <property>
    <name>hive.sentry.provider.resource</name>
    <value>hdfs://cdh54kerldaphdfs/user/secconfig/sentry/sentry-provider.ini</value>
    <!--
       If the hdfs-site.xml points to HDFS, the path will be in HDFS;
       alternatively you could specify a full path, e.g.:
       hdfs://namenode:port/path/to/authz-provider.ini
       file:///path/to/authz-provider.ini
    -->
  </property>

  <property>
      <name>sentry.hive.server</name>
      <description>must be same on Impala and Hive</description>
      <value>server1</value>
  </property>
</configuration>
 
In this setup, I have a KDC + openldap backend + HA NNs/JTs and when I used beeline to connect, it seems to work initially but I have a feeling that this does  not look right as
  <property>
    <name>hive.sentry.conf.url</name>
    <value>file:///etc/sentry/conf/sentry-site.xml</value>
  </property
 
could not be found => most likely sentry is not going to work.
 
 
    [root@cdh54-kerldap-hive hive]# kinit mydomain
    Password for user1@MYDOMAINDEV.COM:
    [root@cdh54-kerldap-hive hive]# beeline --verbose
    Beeline version 1.1.0-cdh5.4.0 by Apache Hive
    beeline> !connect jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM
    scan complete in 2ms
    Connecting to jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM
    Enter username for jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM: hiveuser
    Enter password for jdbc:hive2://localhost:10000/default;principal=hive/cdh54-kerldap-hive.mydomaindev.com@MYDOMAINDEV.COM: ********
    Connected to: Apache Hive (version 1.1.0-cdh5.4.0)
    Driver: Hive JDBC (version 1.1.0-cdh5.4.0)
    Transaction isolation: TRANSACTION_REPEATABLE_READ
    0: jdbc:hive2://localhost:10000/default> show tables;
    Getting log thread is interrupted, since query is done!                     
    +-----------+--+
    | tab_name  |
    +-----------+--+
    +-----------+--+
    No rows selected (3.683 seconds)
 
I turned logging to DEBUG level and here are what I found in hive-metastore.out. Here is the pastebin to this file http://pastebin.com/jZMcJmyJ
Here is the pastebin to hive-server2.out http://pastebin.com/1YbaNjz4.
 
From the log of hive-server2.out, it looks like it actually work because of these lines:
 
15/06/01 15:24:04 INFO hive.metastore: Mestastore configuration hive.security.authorization.manager changed from org.apache.hadoop.hive.ql.security.authorization.DefaultHiveAuthorizationProvider to org.apache.sentry.binding.hive.HiveAuthzBindingSessionHook$SentryHiveAuthorizerFactory
....
15/06/01 15:24:04 INFO file.SimpleFileProviderBackend: Parsing hdfs://cdh54kerldaphdfs/user/secconfig/sentry/sentry-provider.ini
15/06/01 15:24:04 INFO file.SimpleFileProviderBackend: Filesystem: hdfs://cdh54kerldaphdfs
15/06/01 15:24:04 INFO file.PolicyFiles: Opening hdfs://cdh54kerldaphdfs/user/secconfig/sentry/sentry-provider.ini
 
Please shed some lights on why there are WARN messages for these two properties.
 
Thank you very much for all your helps.
Steven.