question Re: How to can access the HDFS using my local linux user(permission denied) in Support Questions

How to can access the HDFS using my local linux user(permission denied)

Mondi — Fri, 16 Sep 2022 14:36:37 GMT

I'm using CDH 6.3.2. When trying to create directories in hdfs, I'm having permission denied like this one:

[devuser@master01 ~]$ hdfs dfs -mkdir /dev
mkdir: Permission denied: user=devuser, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

How can I add my user on hdfs? and how can my user run jobs as well?

Re: How to can access the HDFS using my local linux user(permission denied)

hughatoclc — Mon, 04 May 2020 12:41:39 GMT

Hi, Mondi!

The msg you are getting is telling that user devuser does not have permission to write to the root (/) HDFS folder. This is normal, and just like a Linux FS. If you create a folder in HDFS and make your local unix account the owner, you can use that folder as you.

If you want to create objects in hdfs /, you need to become the superuser, hdfs. Example:

# create your folder $ sudo -u hdfs hdfs dfs -mkdir /user/devuser # make devuser the owner: $ sudo -u hdfs hdfs -chown devuser /user/devuser

You first elevate to hdfs, then run the hdfs dfs command. This wil create your personal directory in /user and make you the owner. Now, when you run hdfs commands as devuser, you will have permission to modify objects in that folder.

CAREFUL: elevating to hdfs is like becoming root in linux. You can do a lot of damage if you make a mistake.

Re: How to can access the HDFS using my local linux user(permission denied)

Mondi — Tue, 05 May 2020 02:41:06 GMT

Hi @hughatoclc thanks for this. this is a big help. however, how can I make my user also in hdfs supergroup? so that I can freely explore the hdfs and also can I run jobs using this as well?

Re: How to can access the HDFS using my local linux user(permission denied)

rachid-berkane — Tue, 05 May 2020 08:34:16 GMT

@Mondi

just update dfs.cluster.administrators with the admin username you want in hdfs config (and restart hdfs,yarn,MR2,... services)

ex: dfs.cluster.administrators = hdfs,ops

you can also use a HDFS administrators group (only one administrator group) using dfs.permissions.superusergroup

ex : dfs.permissions.superusergroup = operations

to verify config has been updated, once services restarted

hdfs getconf -confKey dfs.cluster.administrators

hdfs getconf -confKey dfs.permissions.superusergroup