Created 01-08-2018 05:11 AM
Dear all,
I have enabled ACL on the ambari console and restarted the required services and I'm able to set the permissions for specific group as well. But when they try to execute it is not working. Need your suggestions. My HDP version is 2.4 and hadoop 2.7.
getfacl permission on the folder and file is:
$ hdfs dfs -getfacl -R /abc/month=12/
# file: /abc/month=12
# owner: abiuser
# group: dfsusers
user::rwx
group::r-x
group:data_team:r--
mask::r-x
other::---
default:user::rwx
default:group::r-x
default:group:data_team:r-x
default:mask::r-x
default:other::---
# file: /abc/month=12/file1.bcsf
# owner: abiuser
# group: dfsusers
user::rwx
group::r--
group:data_team:r--
mask::r--
other::---
user A and B are part of data_team, when they try to read the file we are getting the below error.
$ hadoop fs -ls /abc/month=12
ls: Permission denied: user=A, access=EXECUTE, inode="/abc/month=12":abiuser:dfsusers:drwxrwx---
Appreciate any suggestion / help?
Thank you
Created 01-11-2018 04:00 PM
I have successfully reproduced as you request "created a different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission?"
Created file acltest2.txt as user abiuser see contents
[root@nakuru ~]# su - abiuser [abiuser@nakuru ~]$ vi acltest2.txt Could you create different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission? Thank you.
Check the file
[abiuser@nakuru ~]$ ls -al -rw-r--r-- 1 abiuser dfsusers 151 Jan 11 13:00 acltest2.txt
Copied the file to hdfs
[abiuser@nakuru ~]$ hdfs dfs -put acltest2.txt /abc/month=12
Confirmation of file in HDFS note user and group
[abiuser@nakuru ~]$ hdfs dfs -ls /abc/month=12 Found 2 items -rw-r--r-- 3 abiuser dfsusers 151 2018-01-11 13:00 /abc/month=12/acltest2.txt -rw-r--r-- 3 abiuser dfsusers 249 2018-01-11 12:38 /abc/month=12/file1.txt
Set the ACL for group data_team [readonly] where usera and userb belong
[abiuser@nakuru ~]$ hdfs dfs -setfacl -m group:data_team:r-- /abc/month=12/acltest2.txt
Changed to usera
[root@nakuru ~]# su - usera
Successfully read the file as usera
[usera@nakuru ~]$ hdfs dfs -cat /abc/month=12/actest2.txt Could you create different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission? Thank you.
Now lets check the ACL's
[usera@nakuru ~]$ hdfs dfs -getfacl -R /abc/month=12/ # file: /abc/month=12 # owner: abiuser # group: dfsusers user::rwx group::r-x other::r-x # file: /abc/month=12/acltest2.txt # owner: abiuser # group: dfsusers user::rw- group::r-- group:data_team:r-- group:dfsusers:r-- mask::r-- other::r-- # file: /abc/month=12/file1.txt # owner: abiuser # group: dfsusers user::rw- group::r-- other::r--
Hope that answers your issue where did you encounter the problem is there a step you missed?
Please accept and close this thread
Created 01-08-2018 08:54 AM
Please, could you explain the steps you did? To reproduce your scenario can you elaborate is user A and B =abiuser? what is the relation between dfsusers and data_team.
I have implemented numerous variations of permissions and I don't see why this shouldn't work
Created 01-10-2018 09:27 AM
1. ACL feature is enabled by adding the below entry in custom hdfs-site.xml file and restarted the required services from ambari console.
<property>
<name>dfs.namenode.acls.enabled</name>
<value>true</value>
</property>
2. I gave sample as A and B user and they have been added to the group data_team (on Linux level), they are not abiuser. abiuser is the owner of the file. dfsusers is the group of that file (/abc/month=12/file1.bcsf). ACL permission added for the group data_team using the below command.
hdfs dfs -setfacl -m -R group:data_team:r-x /abc/month=12/
3. Above setup is done, but still user A and B not able to read or access the files where ACL permission been given.
Created 01-10-2018 10:17 AM
As suggested by Geoffery enable your setting for ACL to make it working. Furthermore,this is the link from HDP site regarding ACLs. It covers all basics how to enable, setup and check ACLs on Hadoop.
Created 01-11-2018 02:31 AM
Yes, I have referred those documents already and set as required. Problem is it is not allowing the user to read the file which got a proper permission in ACL. You may go through my initial postings with the steps.
Thank you
Created 01-10-2018 10:36 PM
I have tried to reproduce your environment as below. HDP 2.6.2 Ambari 2.5.2, I don't think the version difference is an issue. Created group data_team,dfusers and users abisuer,usera and userb,please try to follow the steps I used to understand and compare with your own. I set the dfs.namenode.acls.enabled to true using Ambari which is the recommended way.
Created groups and users
[root@nakuru ~]# groupadd data_team [root@nakuru ~]# useradd -G data_team usera [root@nakuru ~]# useradd -G data_team userb [root@nakuru ~]# groupadd dfsusers [root@nakuru ~]# useradd abiuser
Switched to user abiuser belonging to group dfsusers and created a file file1.txt with the below contents
[root@nakuru ~]# su - abiuser [abiuser@nakuru ~]$ vi file1.txt /*contents*/ I have enabled ACL on the ambari console and restarted the required services and I'm able to set the permissions for specific group as well. But when they try to execute it is not working. Need your suggestions. My HDP version is 2.4 and hadoop 2.7.
Checked the saved file
[abiuser@nakuru ~]$ ls -al -rw-r--r-- 1 abiuser abiuser 250 Jan 10 22:24 file1.txt
Enabled ACL (custom hdfs-site.xml) through Ambari.
dfs.namenode.acls.enabled=true
Restart all stale configs in my case
HDFS
YARN
MapReduces2
Atlas
As hdfs user created the directory and change ownership and permission
[hdfs@nakuru ~]$ hdfs dfs -mkdir -p /abc/month=12 [hdfs@nakuru ~]$ hdfs dfs -chown -R abiuser:dfsusers /abc/month=12
Validate the above
[hdfs@nakuru ~]$ hdfs dfs -ls /abc Found 1 items drwxr-xr-x - abiuser dfsusers 0 2018-01-10 22:40 /abc/month=12
Copy the file1.txt from local to hdfs
[abiuser@nakuru ~]$ hdfs dfs -put file1.txt /abc/month=12 [abiuser@nakuru ~]$ hdfs dfs -ls /abc/month=12 Found 1 items -rw-r--r-- 3 abiuser dfsusers 250 2018-01-10 22:46 /abc/month=12/file1.txt
Now see the ACL's on the file
[abiuser@nakuru ~]$ hdfs dfs -getfacl -R /abc/month=12/ # file: /abc/month=12 # owner: abiuser # group: dfsusers user::rwx group::r-x other::r-x # file: /abc/month=12/file1.txt # owner: abiuser # group: dfsusers user::rw- group::r-- other::r--
Now set the ACL rwx for usera and userb as the file owner abiuser
[abiuser@nakuru ~]$ hdfs dfs -setfacl -m user:usera:rwx /abc/month=12/file1.txt [abiuser@nakuru ~]$ hdfs dfs -setfacl -m user:userb:rwx /abc/month=12/file1.txt
Validate the above ACL's for the file1.txt
[abiuser@nakuru ~]$ hdfs dfs -getfacl -R /abc/month=12/ # file: /abc/month=12 # owner: abiuser # group: dfsusers user::rwx group::r-x other::r-x # file: /abc/month=12/file1.txt # owner: abiuser # group: dfsusers user::rw- user:usera:rwx user:userb:rwx group::r-- mask::rwx other::r--
See if usera can read the file
[root@nakuru ~]# su - usera [usera@nakuru ~]$ hdfs dfs -cat /abc/month=12/file1.txt I have enabled ACL on the ambari console and restarted the required services and I'm able to set the permissions for specific group as well. But when they try to execute it is not working. Need your suggestions. My HDP version is 2.4 and hadoop 2.7.
I get exactly the contents of the file1.txt
See if userb can read the file
[root@nakuru ~]# su - userb [userb@nakuru ~]$ hdfs dfs -cat /abc/month=12/file1.txt I have enabled ACL on the ambari console and restarted the required services and I'm able to set the permissions for specific group as well. But when they try to execute it is not working. Need your suggestions. My HDP version is 2.4 and hadoop 2.7.
Voila , that answers your question the file owner and group is abiuser:dfsuser but usera and userb from a different group data_team can successfully read the file1.txt
Could you Accept the answer by Clicking on Accept button below, if this answers your problem that would be great help to Community users to find solution quickly.
Created 01-11-2018 02:29 AM
First of all thanks for your time and outputs, samething been done with only one difference. I have given acl permission for the group data_team with r-x instead of individual users. In future there will be a requirement for other users to get only read access which I can do by just adding them to the group data_team in Linux. Hope this also should work. Below is the command I have used.
hdfs dfs -setfacl -m -R group:data_team:r-x /abc/month=12
Could you create different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission?
Thank you.
Created 01-11-2018 04:00 PM
I have successfully reproduced as you request "created a different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission?"
Created file acltest2.txt as user abiuser see contents
[root@nakuru ~]# su - abiuser [abiuser@nakuru ~]$ vi acltest2.txt Could you create different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission? Thank you.
Check the file
[abiuser@nakuru ~]$ ls -al -rw-r--r-- 1 abiuser dfsusers 151 Jan 11 13:00 acltest2.txt
Copied the file to hdfs
[abiuser@nakuru ~]$ hdfs dfs -put acltest2.txt /abc/month=12
Confirmation of file in HDFS note user and group
[abiuser@nakuru ~]$ hdfs dfs -ls /abc/month=12 Found 2 items -rw-r--r-- 3 abiuser dfsusers 151 2018-01-11 13:00 /abc/month=12/acltest2.txt -rw-r--r-- 3 abiuser dfsusers 249 2018-01-11 12:38 /abc/month=12/file1.txt
Set the ACL for group data_team [readonly] where usera and userb belong
[abiuser@nakuru ~]$ hdfs dfs -setfacl -m group:data_team:r-- /abc/month=12/acltest2.txt
Changed to usera
[root@nakuru ~]# su - usera
Successfully read the file as usera
[usera@nakuru ~]$ hdfs dfs -cat /abc/month=12/actest2.txt Could you create different file with abiuser as owner and dfsusers as group and add ACL for the group data_team with just read permission? Thank you.
Now lets check the ACL's
[usera@nakuru ~]$ hdfs dfs -getfacl -R /abc/month=12/ # file: /abc/month=12 # owner: abiuser # group: dfsusers user::rwx group::r-x other::r-x # file: /abc/month=12/acltest2.txt # owner: abiuser # group: dfsusers user::rw- group::r-- group:data_team:r-- group:dfsusers:r-- mask::r-- other::r-- # file: /abc/month=12/file1.txt # owner: abiuser # group: dfsusers user::rw- group::r-- other::r--
Hope that answers your issue where did you encounter the problem is there a step you missed?
Please accept and close this thread
Created 01-14-2018 10:59 PM
@Muthukumar S
Can you revert whether this issue was resolved if so then accept and close the thread.
Created 01-15-2018 03:14 PM
Sorry was stuck up in few issues and missed to reply. Yes the steps you have mentioned all followed. I was getting the error which i have shown in my first post. Hence i initialized this thread and you have provided the same steps which i have followed. Not sure what is wrong or some bug ?
When ran with user A or B who are part of data_team in ACL. 😞
"$ hadoop fs -ls /abc/month=12
ls: Permission denied: user=A, access=EXECUTE, inode="/abc/month=12":abiuser:dfsusers:drwxrwx---"