question Re: Ranger policy for Hive table backed on HBase in Archives of Support Questions (Read Only)

Ranger policy for Hive table backed on HBase

geko — Mon, 08 Feb 2016 23:38:08 GMT

Hi,

I have a Hive table which sits on top of HBase and create two policies for the same user in Ranger. One for Hive and one for HBase, to allow access to the corresponding table.

In Ranger I can see the agents has successfully registered and they received the latest changes.

If I now do a select * from hivetableonhbase; vie Hue I receive the error:

java.io.IOException: org.apache.hadoop.hbase.security.AccessDeniedException: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'pklfsvc' for scanner open on table hbaseidv
	at com.xasecure.authorization.hbase.XaSecureAuthorizationCoprocessor.preScannerOpen(XaSecureAuthorizationCoprocessor.java:719)
	at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preScannerOpen(RegionCoprocessorHost.java:1870)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3167)
	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29994)
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
	at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
	at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)

at java.lang.Thread.run(Thread.java:745)

And in addition I do not see any attempt to access HBase in Ranger Audit log.

Is there something special in Accessing HBase via Hive with respect to grant permissions to users ?!?!

Re: Ranger policy for Hive table backed on HBase

nsabharwal — Mon, 08 Feb 2016 23:41:29 GMT

@Gerd Koenig

This is great question and personally, I have never worked on this use case.

Did you grant access to hive and hbase tables to user pklfsvc in Hive and HBase policies?

Re: Ranger policy for Hive table backed on HBase

geko — Mon, 08 Feb 2016 23:44:14 GMT

Hello @Neeraj Sabharwal , yes, user 'pklfsvc' has rwx permissions in Hive- and HBase-Ranger policy

Re: Ranger policy for Hive table backed on HBase

nsabharwal — Mon, 08 Feb 2016 23:45:33 GMT

@Gerd Koenig Are you able to access using beeline?

Re: Ranger policy for Hive table backed on HBase

aervits — Mon, 08 Feb 2016 23:55:38 GMT

@Gerd Koenig

awesome question, according to this jira, can you double check the znode information for the table? jira this might be a bug with Ranger.

scan 'hbase:acl'

Re: Ranger policy for Hive table backed on HBase

geko — Tue, 09 Feb 2016 00:27:03 GMT

@Neeraj Sabharwal , connect yes, but also permission error:

0: jdbc:hive2://b0d02ef2:10> show tables;
+----------------------+--+
|       tab_name       |
+----------------------+--+
| hbaseidvtmp  |
| hbaseidv    |

2 rows selected (0.293 seconds)
0: jdbc:hive2://b0d02ef2:10> select * from hbaseidv;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [pklfsvc] does not have [SELECT] privilege on [<dbname>/hbaseidv/birthdate] (state=42000,code=40000)
0: jdbc:hive2://b0d02ef2:10>

Re: Ranger policy for Hive table backed on HBase

nsabharwal — Tue, 09 Feb 2016 00:28:49 GMT

@Gerd Koenig Perfect!!!! 🙂

So...Hue is casuing the trouble?

Re: Ranger policy for Hive table backed on HBase

geko — Tue, 09 Feb 2016 00:33:46 GMT

Hi @Artem Ervits , please find below the output of your command. Seems like there are no settings for table 'hbaseidv' ...

ROW                                    COLUMN+CELL
 ambarismoketest                       column=l:ambari-qa, timestamp=1453802112798, value=RWXCA
 hbase:acl                             column=l:ambari-qa, timestamp=1453802098747, value=RWXCA
2 row(s) in 0.5710 seconds

Do I have to set something directly in HBase ?

My assumption was that Ranger-HBase-policy will abstract this, like for HDFS (HDFS-ACL set to 000 and grant access via Ranger ) ?!?!

Re: Ranger policy for Hive table backed on HBase

aervits — Tue, 09 Feb 2016 00:42:37 GMT

@Gerd Koenig are you following similar steps as this guide? You should be able to see the table from both places https://community.hortonworks.com/content/kbentry/14806/working-with-hbase-and-hive-wip.html

Re: Ranger policy for Hive table backed on HBase

geko — Tue, 09 Feb 2016 00:50:12 GMT

@Neeraj Sabharwal , nope, Hue is not causing the troubles since via Beeline I receive the same permission denied error ...

Re: Ranger policy for Hive table backed on HBase

nsabharwal — Tue, 09 Feb 2016 00:52:00 GMT

@Gerd Koenig birthdate is the only column in picture?

Re: Ranger policy for Hive table backed on HBase

geko — Tue, 09 Feb 2016 00:59:41 GMT

@Artem Ervits , thanks for this great link.

If I connect as user 'hbase' I can execute a "scan 'hbaseidv' " successfully, but if I open a hbase shell as user pklfsvc I receive the error shown below.

Do I have to grant rwx to that user on HBase level before putting Ranger policies on top ?

hbase(main):002:0> scan 'hbaseidv'
ROW                                         COLUMN+CELL
ERROR: org.apache.hadoop.hbase.security.AccessDeniedException: Insufficient permissions for user 'pklfsvc' for scanner open on table hbaseidv

Re: Ranger policy for Hive table backed on HBase

aervits — Tue, 09 Feb 2016 01:02:33 GMT

@Gerd Koenig the only advice I have for you as this is a unique use case is to try and then post an article :).

Re: Ranger policy for Hive table backed on HBase

geko — Tue, 09 Feb 2016 01:38:18 GMT

@Artem Ervits , sure, will do so as soon as I have prepared the stuff...

Re: Ranger policy for Hive table backed on HBase

geko — Wed, 10 Feb 2016 21:23:19 GMT

Hi @Artem Ervits , @Neeraj Sabharwal ,

at the end, using Ranger policies for Hive-on-top-of-HBase works as supposed to do so, by defining Hive-Policy and HBase-Policy for the involved tables.

The issue I had was the following, although I really don't understand why it is like it is:

switching back to Ranger-HTTP from HTTPS left the policy_mgr_url starting with HTTPS://<ranger-admin>:<port>; on the HBase-REGIONSERVERS, thereby the REGIONSERVERS were complaining that they cannot grab latest Ranger policies due to SSL error. This was the reason why my HBase policies were never applied, because they never got fetched by the REGIONSERVERS.

Now the point that is confusing me:

why the REGIONSERVERS ???? On the HBase-Master nodes there was no error, they had received the latest HBase-policies and therefore in the Ranger-Audit the agents heartbeat has been updated (and therefore I thought everything's fine).

Isn't it the similar behaviour of Ranger-plugin like in HDFS, that the plugin just hooks into the "master"-process Namenode , what is the role of Ranger-in-Regionserver here ?