HDFS folder permission denied, but the user is in the owner group.

[#1]

I am using HDP. The inode in the following code is a managed hive table.

# id zeppelin
uid=1017(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)
# sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1
ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive/test1":hive:hadoop:drwxrwx---

The user zeppelin is in hadoop group, which has full permisison on the hdfs folder. So why do I get the permission error?

[#2 · Accepted solution]

@Seaport,

Please refer to documentation here:
https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setfacl

Maybe try:

sudo -u hdfs hadoop fs -setfacl -m group:hadoop:r-x /warehouse/tablespace/managed/hive/test1

[#3]

@Seaport ,

Can you try:

hdfs groups zeppelin

Or run "id zeppelin" on the active NN host?

[#4]

# hdfs groups zeppelin
zeppelin : hadoop zeppelin

On the name node, 

# id zeppelin
uid=1018(zeppelin) gid=1003(hadoop) groups=1003(hadoop),1005(zeppelin)

[#5]

I might have found the reason.

I ran the following command as hdfs, which is the superuser of hdfs.

$ hadoop fs -getfacl /warehouse/tablespace/managed/hive/test1
# file: /warehouse/tablespace/managed/hive/test1
# owner: hive
# group: hadoop
user::rwx
user:hive:rwx
group::---
mask::rwx
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

The output, as I understand, shows that the group owner has no permission on the folder. My guess is that, HDP Hive uses ACL to limit direct access to files behind managed tables. HDP Hive tries to force accessing to managed tables only through Hive.

[#6]

@Seaport

Great, thanks for sharing! So try to use "hdfs dfs -setfacl" to update it and see how it goes.

[#7]

I tried the following command

# sudo -u hdfs hadoop fs -setfacl -m g::rx /warehouse/tablespace/managed/hive/test1

But I got the error

-setfacl: Invalid type of acl in <aclSpec> :g::rx

The acl spec is to modify the owning group permission to rx.

Any suggestion?

[#8 · Accepted solution]

@Seaport,

Please refer to documentation here:
https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setfacl

Maybe try:

sudo -u hdfs hadoop fs -setfacl -m group:hadoop:r-x /warehouse/tablespace/managed/hive/test1

[#9 · Accepted solution]

@Seaport 

As the permission is with the zeppelin user  [other] you will need to do that at a user level, remember fine-grained security ONLY give what is necessary !!

$ hdfs dfs -getfacl /warehouse/tablespace/managed/hive
# file: /warehouse/tablespace/managed/hive
# owner: hive
# group: hadoop
user::rwx
group::---
other::---
default:user::rwx
default:user:hive:rwx
default:group::---
default:mask::rwx
default:other::---

The command below will set [ r-x } bits to  the correct ACL you can change to rwx if you wish

hdfs dfs -setfacl -R -m user:zeppelin:r-x /warehouse/tablespace/managed/hive

Thereafter the zeppelin user can 

[zeppelin~]$ hdfs dfs -ls /warehouse/tablespace/managed/hive
Found 3 items
drwxrwx---+  - hive hadoop          0 2018-12-12 23:42 /warehouse/tablespace/managed/hive/information_schema.db
drwxrwx---+  - hive hadoop          0 2018-12-12 23:41 /warehouse/tablespace/managed/hive/sys.db
drwxrwx---+  - hive hadoop          0 2020-01-15 00:20 /warehouse/tablespace/managed/hive/zepp.db

The earlier error is gone 

ls: Permission denied: user=zeppelin, access=READ_EXECUTE, inode="/warehouse/tablespace/managed/hive":hive:hadoop:drwx------

Happy hadooping

[#10]

@Shelton @EricL Thank you both.

the correct ACL spec is group::r-x

Now the following command works.

sudo -u zeppelin hadoop fs -ls /warehouse/tablespace/managed/hive/test1

From what I just ran into, I feel that, by design, Hive takes extra effort to prevent users from accessing managed table files directly. I will follow that design and access Hive managed table only through Hive.