Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can sentry provide security over Hive External Tables.? If yes, how to do that.?

Highlighted

Can sentry provide security over Hive External Tables.? If yes, how to do that.?

Explorer

Hi, 

 

We have data in HDFS and use linux security (file level access control) to authorize access to those files. We have external tables defined in Hive for that data. However, we see that there is no security defined at Hive level and the data through Hive is viewable by all.

 

Can sentry help us.? Can we creae role based access over Hive External tables.?

I know it does well for Impala tables, but not sure about Hive External tables. 

 

If it can do that, Is there any guidelines for achieving it.?

10 REPLIES 10

Re: Can sentry provide security over Hive External Tables.

Master Guru
I assume by "linux security (file level access control" you meant HDFS ACLs
[1].

Sentry can control authorisation over external table objects in Hive just
as it would for any other table; however, for Sentry HDFS ACLs feature [2]
to cover external paths you would need to add the parent of such paths to
the HDFS Configuration field called "Sentry Synchronization Path Prefixes".

[1] -
http://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.h...
[2] -
http://www.cloudera.com/documentation/enterprise/latest/topics/sg_hdfs_sentry_sync.html

Re: Can sentry provide security over Hive External Tables.

Explorer

Hi Harsh

 

Thanks for you response. We are facing an issue with security and If you could help, it would be great. 

 

We do not have sentry yet with our client, and planning to go that path later.

How could we set security over HDFS Data and Hive external tables with ACLs.? We would like to restrict the read access on the data and hive external tables. 

 

We have enabled acls property in hdfs-site.xml and hive-site.xml and removed the read access to all the other users. But, still hdfs dfs -cat and hive select * are working. Could not figure out why.

Re: Can sentry provide security over Hive External Tables.

Master Guru
Yes you can self-manage ACLs if you don't prefer to use Sentry (although it
could get tedious)

What are your ACLs, and how are you testing for access? Can you post more
details with appropriate setfacl/getfacl outputs and the klist output?

Re: Can sentry provide security over Hive External Tables.

Explorer

Hi Harsh,

 

This is what we have tried.

 

we have enabled dfs.namenode.acls.enabled in hdfs-site.xml and we have added acls property for hive-site.xml and restarted the cluster. 

 

We have made sure the folder and file in hdfs has 600 (rw-,---,---) in permissions. 

This is how the facls for that test folder are.

 

# file: /user/ravi/sec

# owner: ravi

# group: hdfs

user::r-x

group::---

other::---

 

# file: /user/ravi/sec/test.tsv

# owner: ravi

# group: hdfs

user::r-x

user:auth_test:---

group::---

mask::---

other::---

 

Even though, auth_test does not have any permissions, we are still able to get data using -cat and hive external table.

 

What was the mistake in our approach.?

Re: Can sentry provide security over Hive External Tables.

Explorer

Hi Harsh,

 

i have double checked and did as this.

 

[sdaruna@airisdata~]$ sudo -u hdfs hdfs dfs -setfacl -R --set user::rw-,user:hadoop:---,group::---,other::--- /user/ravi/sec

 

[sdaruna@airisdata ~]$ hdfs dfs -getfacl -R /user/ravi/sec

# file: /user/ravi/sec

# owner: ravi

# group: hdfs

user::rw-

user:hadoop:---

group::---

mask::---

other::---

 

# file: /user/ravi/sec/test.tsv

# owner: ravi

# group: hdfs

user::rw-

user:hadoop:---

group::---

mask::---

other::---

 

Now, with the test user (auth_test), that we have created, here is how his groups

 

[sdaruna@airisdata ~]$ groups auth_test

auth_test : auth_test auth_test_gp

 

He does not belongs to hadoop group. 

 

But, when we login with that user, we are able to perform hdfs -cat and get data and same with hive external table.

Re: Can sentry provide security over Hive External Tables.

Master Guru
This is odd. Are you certain the login is done right (for auth_test)? The
klist output does show its principal credentials (assuming you use
Kerberos/AD security)?

If yes, then could you tell if your regular permissions even work? Leaving
aside ACLs, does simple permission mode and ownership rules ever give you
access denied errors? If not, you may have dfs.permissions,enabled set to
false in your NameNode configs.

Re: Can sentry provide security over Hive External Tables.

Explorer

Hi Harsh, 

 

Is Kerberos/AD is must for storage based ACLs to work.? I am sorry, we do not have that in developement cluster and we are playing as per what i have mentioned to see if that works.

 

Is kerberos mandate.?

Re: Can sentry provide security over Hive External Tables.

Master Guru
While not mandatory, it would only make sense to me to apply authorization
(via ACLs) if strong authentication is first in place. Otherwise, any user
can impersonate any other via the env-var HADOOP_USER_NAME or other manner,
and void all your prevention guarantees anyway.

That said, a simple ACL setup should work if permissions config is in
effect. Could you confirm?

Re: Can sentry provide security over Hive External Tables.

Explorer

Hi Harsh,

 

Yes it is enabled. Otherwise, the setfacls will return an error saying, it will not allow when dfs.namenode.acls.enabled is false. We have got that error at the start, and then we have modified the value. I checked it now. (in cloudera manager, when i entered into cluster and then to HDFS, and then to configuration, i searched for dfs.namenode.acls.enabled, i got the property in Service-Wide / Security category and value is checked)

 

We have restarted the HDFS services and then it allowed to do the setfacls. 

We are ok for the impersonate case, but other user without using any sudo or impersonate, able to do hdfs -cat.

 

Do you see us missing any thing.? I can send you screen shots of the configuration if you need, if you can provide your mail id.

 

Our customer is stressing on having security enabled and we tried every thing to see it working. but with no success.