Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

HDFS folder permission denied, but the user is in the owner group.

avatar
Expert Contributor

I am using HDP. The inode in the following code is a managed hive table.

 

# id zeppelin
uid=1017(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)
# sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1
ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive/test1":hive:hadoop:drwxrwx---

 

The user zeppelin is in hadoop group, which has full permisison on the hdfs folder. So why do I get the permission error?

2 ACCEPTED SOLUTIONS

avatar
Super Guru
@Seaport,

Please refer to documentation here:
https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setfacl

Maybe try:

sudo -u hdfs hadoop fs -setfacl -m group:hadoop:r-x /warehouse/tablespace/managed/hive/test1

View solution in original post

avatar
Master Mentor

@Seaport 

As the permission is with the zeppelin user  [other] you will need to do that at a user level, remember fine-grained security ONLY give what is necessary !!

$ hdfs dfs -getfacl /warehouse/tablespace/managed/hive
# file: /warehouse/tablespace/managed/hive
# owner: hive
# group: hadoop
user::rwx
group::---
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

The command below will set [ r-x } bits to  the correct ACL you can change to rwx if you wish

hdfs dfs -setfacl -R -m user:zeppelin:r-x /warehouse/tablespace/managed/hive

 

Thereafter the zeppelin user can 

 

[zeppelin~]$ hdfs dfs -ls /warehouse/tablespace/managed/hive
Found 3 items
drwxrwx---+  - hive hadoop          0 2018-12-12 23:42 /warehouse/tablespace/managed/hive/information_schema.db
drwxrwx---+  - hive hadoop          0 2018-12-12 23:41 /warehouse/tablespace/managed/hive/sys.db
drwxrwx---+  - hive hadoop          0 2020-01-15 00:20 /warehouse/tablespace/managed/hive/zepp.db

The earlier error is gone 


ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive":hive:hadoop:drwx------

 

Happy hadooping

 

View solution in original post

8 REPLIES 8

avatar
Super Guru
@Seaport ,

Can you try:

hdfs groups zeppelin

Or run "id zeppelin" on the active NN host?

avatar
Expert Contributor

# hdfs groups zeppelin
zeppelin : hadoop zeppelin

 

On the name node, 

# id zeppelin
uid=1018(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)

avatar
Expert Contributor

I might have found the reason.

 

I ran the following command as hdfs, which is the superuser of hdfs.

$ hadoop fs -getfacl /warehouse/tablespace/managed/hive/test1
# file: /warehouse/tablespace/managed/hive/test1
# owner: hive
# group: hadoop
user::rwx
user:hive:rwx
group::---
mask::rwx
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

 

The output, as I understand, shows that the group owner has no permission on the folder. My guess is that, HDP Hive uses ACL to limit direct access to files behind managed tables. HDP Hive tries to force accessing to managed tables only through Hive.

avatar
Super Guru
@Seaport

Great, thanks for sharing! So try to use "hdfs dfs -setfacl" to update it and see how it goes.

avatar
Expert Contributor

I tried the following command

# sudo -u hdfs hadoop fs -setfacl -m g::rx /warehouse/tablespace/managed/hive/test1

But I got the error

-setfacl: Invalid type of acl in <aclSpec> :g::rx

The acl spec is to modify the owning group permission to rx.

Any suggestion?

avatar
Super Guru
@Seaport,

Please refer to documentation here:
https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setfacl

Maybe try:

sudo -u hdfs hadoop fs -setfacl -m group:hadoop:r-x /warehouse/tablespace/managed/hive/test1

avatar
Master Mentor

@Seaport 

As the permission is with the zeppelin user  [other] you will need to do that at a user level, remember fine-grained security ONLY give what is necessary !!

$ hdfs dfs -getfacl /warehouse/tablespace/managed/hive
# file: /warehouse/tablespace/managed/hive
# owner: hive
# group: hadoop
user::rwx
group::---
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

The command below will set [ r-x } bits to  the correct ACL you can change to rwx if you wish

hdfs dfs -setfacl -R -m user:zeppelin:r-x /warehouse/tablespace/managed/hive

 

Thereafter the zeppelin user can 

 

[zeppelin~]$ hdfs dfs -ls /warehouse/tablespace/managed/hive
Found 3 items
drwxrwx---+  - hive hadoop          0 2018-12-12 23:42 /warehouse/tablespace/managed/hive/information_schema.db
drwxrwx---+  - hive hadoop          0 2018-12-12 23:41 /warehouse/tablespace/managed/hive/sys.db
drwxrwx---+  - hive hadoop          0 2020-01-15 00:20 /warehouse/tablespace/managed/hive/zepp.db

The earlier error is gone 


ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive":hive:hadoop:drwx------

 

Happy hadooping

 

avatar
Expert Contributor

@Shelton @EricL Thank you both.

the correct ACL spec is group::r-x

Now the following command works.

sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1

 

From what I just ran into, I feel that, by design, Hive takes extra effort to prevent users from accessing managed table files directly. I will follow that design and access Hive managed table only through Hive.