Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDFS folder permission denied, but the user is in the owner group.

Solved Go to solution
Highlighted

HDFS folder permission denied, but the user is in the owner group.

Contributor

I am using HDP. The inode in the following code is a managed hive table.

 

# id zeppelin
uid=1017(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)
# sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1
ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive/test1":hive:hadoop:drwxrwx---

 

The user zeppelin is in hadoop group, which has full permisison on the hdfs folder. So why do I get the permission error?

2 ACCEPTED SOLUTIONS

Accepted Solutions
Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Guru
@Seaport,

Please refer to documentation here:
https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setfacl

Maybe try:

sudo -u hdfs hadoop fs -setfacl -m group:hadoop:r-x /warehouse/tablespace/managed/hive/test1

View solution in original post

Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Mentor

@Seaport 

As the permission is with the zeppelin user  [other] you will need to do that at a user level, remember fine-grained security ONLY give what is necessary !!

$ hdfs dfs -getfacl /warehouse/tablespace/managed/hive
# file: /warehouse/tablespace/managed/hive
# owner: hive
# group: hadoop
user::rwx
group::---
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

The command below will set [ r-x } bits to  the correct ACL you can change to rwx if you wish

hdfs dfs -setfacl -R -m user:zeppelin:r-x /warehouse/tablespace/managed/hive

 

Thereafter the zeppelin user can 

 

[zeppelin~]$ hdfs dfs -ls /warehouse/tablespace/managed/hive
Found 3 items
drwxrwx---+  - hive hadoop          0 2018-12-12 23:42 /warehouse/tablespace/managed/hive/information_schema.db
drwxrwx---+  - hive hadoop          0 2018-12-12 23:41 /warehouse/tablespace/managed/hive/sys.db
drwxrwx---+  - hive hadoop          0 2020-01-15 00:20 /warehouse/tablespace/managed/hive/zepp.db

The earlier error is gone 


ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive":hive:hadoop:drwx------

 

Happy hadooping

 

View solution in original post

8 REPLIES 8
Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Guru
@Seaport ,

Can you try:

hdfs groups zeppelin

Or run "id zeppelin" on the active NN host?
Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Contributor

# hdfs groups zeppelin
zeppelin : hadoop zeppelin

 

On the name node, 

# id zeppelin
uid=1018(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)

Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Contributor

I might have found the reason.

 

I ran the following command as hdfs, which is the superuser of hdfs.

$ hadoop fs -getfacl /warehouse/tablespace/managed/hive/test1
# file: /warehouse/tablespace/managed/hive/test1
# owner: hive
# group: hadoop
user::rwx
user:hive:rwx
group::---
mask::rwx
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

 

The output, as I understand, shows that the group owner has no permission on the folder. My guess is that, HDP Hive uses ACL to limit direct access to files behind managed tables. HDP Hive tries to force accessing to managed tables only through Hive.

Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Guru
@Seaport

Great, thanks for sharing! So try to use "hdfs dfs -setfacl" to update it and see how it goes.
Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Contributor

I tried the following command

# sudo -u hdfs hadoop fs -setfacl -m g::rx /warehouse/tablespace/managed/hive/test1

But I got the error

-setfacl: Invalid type of acl in <aclSpec> :g::rx

The acl spec is to modify the owning group permission to rx.

Any suggestion?

Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Guru
@Seaport,

Please refer to documentation here:
https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setfacl

Maybe try:

sudo -u hdfs hadoop fs -setfacl -m group:hadoop:r-x /warehouse/tablespace/managed/hive/test1

View solution in original post

Highlighted

Re: HDFS folder permission denied, but the user is in the owner group.

Mentor

@Seaport 

As the permission is with the zeppelin user  [other] you will need to do that at a user level, remember fine-grained security ONLY give what is necessary !!

$ hdfs dfs -getfacl /warehouse/tablespace/managed/hive
# file: /warehouse/tablespace/managed/hive
# owner: hive
# group: hadoop
user::rwx
group::---
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

The command below will set [ r-x } bits to  the correct ACL you can change to rwx if you wish

hdfs dfs -setfacl -R -m user:zeppelin:r-x /warehouse/tablespace/managed/hive

 

Thereafter the zeppelin user can 

 

[zeppelin~]$ hdfs dfs -ls /warehouse/tablespace/managed/hive
Found 3 items
drwxrwx---+  - hive hadoop          0 2018-12-12 23:42 /warehouse/tablespace/managed/hive/information_schema.db
drwxrwx---+  - hive hadoop          0 2018-12-12 23:41 /warehouse/tablespace/managed/hive/sys.db
drwxrwx---+  - hive hadoop          0 2020-01-15 00:20 /warehouse/tablespace/managed/hive/zepp.db

The earlier error is gone 


ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive":hive:hadoop:drwx------

 

Happy hadooping

 

View solution in original post

Re: HDFS folder permission denied, but the user is in the owner group.

Contributor

@Shelton @EricL Thank you both.

the correct ACL spec is group::r-x

Now the following command works.

sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1

 

From what I just ran into, I feel that, by design, Hive takes extra effort to prevent users from accessing managed table files directly. I will follow that design and access Hive managed table only through Hive.

Don't have an account?
Coming from Hortonworks? Activate your account here