question Re: HDFS Policy 'resource path' with placeholder - is there some way ? in Support Questions

HDFS Policy 'resource path' with placeholder - is there some way ?

kaliyugantagoni — Sun, 18 Aug 2019 11:57:37 GMT

I am trying to understand and practice Ranger configurations on the local HDP 2.4 sandbox.

The objective is that when the users(existing in some LDAP group) log-in and try to access HDFS, they should be able to access only their 'home directory' and others like tmp.I did the following :

Create a 'home directory' and for the users as well, note, the owner is still hdfs !. Suggestions welcome here e.g: Is this new /userhome dir. desirable or shall I use the existing /user directory ?

[hdfs@sandbox root]$ hdfs dfs -ls /userhome
Found 2 items
drwxr-xr-x   - hdfs hdfs          0 2016-06-27 14:24 /userhome/HBRDMV
drwxr-xr-x   - hdfs hdfs          0 2016-06-27 14:55 /userhome/ojoqcu

Disabled the default HDFS policy and created two new. The 'HadoopEcosystem' to ensure that the Hadoop users can execute their operations smoothly e.g the Hive CLI can be started by Hive user and used without access issues

In the 'RestrictiveHDFS' policy, I intend to achieve the objective. I have provided the 'Resource Path' accordingly

I think it works, can anyone point out what I have missed to verify ?

[ojoqcu@sandbox ~]$ hdfs dfs -ls /userhome/HBRDMV
ls: Permission denied: user=ojoqcu, access=EXECUTE, inode="/userhome/HBRDMV":hdfs:hdfs:drwxr-x---


[ojoqcu@sandbox ~]$ hdfs dfs -mkdir /userhome/ojoqcu/code


[ojoqcu@sandbox ~]$[ojoqcu@sandbox ~]$ hdfs dfs -ls /userhome/ojoqcu
Found 1 item drwxr-xr-x   - ojoqcu hdfs          0 2016-06-28 11:33 /userhome/ojoqcu/code


[ojoqcu@sandbox ~]$ hdfs dfs -ls /userhome/
ls: Permission denied: user=ojoqcu, access=EXECUTE, inode="/userhome":hdfs:hdfs:drwxr-x---

Now, there may be aplenty users who will have their own directories under /userhome but only the single policy 'RestrictiveHDFS' would suffice, however, the 'Resource Path' doesn't provide a way to specify something like /userhome/{user} where {user} can be any of the from the 'Select User' list under 'User and Group Permissions'. How can I achieve this ?

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

dsharma — Tue, 28 Jun 2016 19:20:39 GMT

@Kaliyug wildcards are allowed in hdfs policy , so you can use as i mentioned in the following screenshot:

screen-shot-2016-06-28-at-53952-pm.png

but here i will suggest you to do following steps:

1. do this for all the users:

use user home in hdfs /user/<username> , and change owner to actual user : for ex:

chown HBRDMV:HBRDMV /user/HBRDMV

chmod 700 -R /user/HBRDMV

2. and for tmp you can create a policy with recursive yes , and add all the users there

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

kaliyugantagoni — Tue, 28 Jun 2016 19:29:56 GMT

But my main question is how can I grant access to all the users to only their own /user/{directory} ? Wildcards certainly won't help here, how do I proceed ?

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

dsharma — Tue, 28 Jun 2016 19:40:19 GMT

you will have to create sep. policy for each resource /user/{directory} , for ex. /user/HBRDMV will have sep. policy for user HBRDMV .

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

kaliyugantagoni — Tue, 28 Jun 2016 20:25:28 GMT

That's awful - if there are 100 users per service, those many policies per service need to be created. Is there something being missed or some better ways to do it ?

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

dsharma — Tue, 28 Jun 2016 20:43:55 GMT

so in this case better approach will be make owner of that user home directory to the user , generally same user is the owner of the user home dir.

for ex: chown HBRDMV:HBRDMV /user/HBRDMV

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

emaxwell — Tue, 28 Jun 2016 21:49:08 GMT

@Kaliyug Antagonist

HDFS security is multi-tiered:

Ranger authorization policies are checked first
HDFS ACLs implemented outside of Ranger
HDFS POSIX permissions (e.g. rwxr-xr-x)

So, what you can do for user home directories is to set the POSIX permissions to 700 and make sure the ownership is <username>:hdfs. This will ensure that only the user has access to his/her home directory. You don't need to create a Ranger policy to allow the access for this. You can do the same for the /tmp directory (set permissions to 777).

There are some best practices for securing HDFS with Ranger.

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

kaliyugantagoni — Tue, 28 Jun 2016 22:06:58 GMT

I got the point of HDFS POSIX permissions, however, I couldn't understand 'HDFS ACLs implemented outside of Ranger' - does this mean one that ACL and Ranger are 'mutually exclusive' ? If yes, what is that ACL is doing which Ranger cannot ? Can you check this community thread which suggests that if you use Ranger, you need not work with ACL.

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

emaxwell — Wed, 29 Jun 2016 00:18:18 GMT

@Kaliyug Antagonist

HDFS has the ability to use ACLs (here's a link). If you don't have Ranger, then you can use ACLs to provide finer grained authorization than you can with POSIX permissions. However, if using Ranger, there is more flexibility and you have a single place to manage authorization for all of the components (not just HDFS). So, if you're using Ranger, you don't really need you use HDFS ACLs.

Re: HDFS Policy 'resource path' with placeholder - is there some way ?

robert_jones — Tue, 26 Jul 2016 15:01:13 GMT

ranger-home-directory-policy.png@Kaliyug Antagonist We've found another neat solution to this, using a resource path of the form: "/user/${id}"

Credit to Naveed Hussain, who found it after we moaned a lot about the alternatives.

Screenshot attached.