Support Questions
Find answers, ask questions, and share your expertise

How to can access the HDFS using my local linux user(permission denied)

Explorer

I'm using CDH 6.3.2. When trying to create directories in hdfs, I'm having permission denied like this one:

[devuser@master01 ~]$ hdfs dfs -mkdir /dev
mkdir: Permission denied: user=devuser, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

How can I add my user on hdfs? and how can my user run jobs as well?

2 ACCEPTED SOLUTIONS

Explorer

Hi, Mondi!

The msg you are getting is telling that user devuser does not have permission to write to the root (/) HDFS folder.  This is normal, and just like a Linux FS.  If you create a folder in HDFS and make your local unix account the owner, you can use that folder as you.

 

If you want to create objects in hdfs /, you need to become the superuser, hdfs.  Example:

   

 

 

# create your folder
$ sudo -u hdfs hdfs dfs -mkdir /user/devuser
# make devuser the owner:
$ sudo -u hdfs hdfs -chown devuser /user/devuser

 

 

You first elevate to hdfs, then run the hdfs dfs command.  This wil create your personal directory in /user and make you the owner.  Now, when you run hdfs commands as devuser, you will have permission to modify objects in that folder.

 

CAREFUL: elevating to hdfs is like becoming root in linux.  You can do a lot of damage if you make a mistake.

 

 

View solution in original post

@Mondi 

just update dfs.cluster.administrators with the admin username you want in hdfs config (and restart hdfs,yarn,MR2,... services)

ex: dfs.cluster.administrators = hdfs,ops

 

you can also use a HDFS administrators group (only one administrator group) using dfs.permissions.superusergroup

ex : dfs.permissions.superusergroup = operations

 

to verify config has been updated, once services restarted

hdfs getconf -confKey dfs.cluster.administrators

or

hdfs getconf -confKey dfs.permissions.superusergroup

 

View solution in original post

3 REPLIES 3

Explorer

Hi, Mondi!

The msg you are getting is telling that user devuser does not have permission to write to the root (/) HDFS folder.  This is normal, and just like a Linux FS.  If you create a folder in HDFS and make your local unix account the owner, you can use that folder as you.

 

If you want to create objects in hdfs /, you need to become the superuser, hdfs.  Example:

   

 

 

# create your folder
$ sudo -u hdfs hdfs dfs -mkdir /user/devuser
# make devuser the owner:
$ sudo -u hdfs hdfs -chown devuser /user/devuser

 

 

You first elevate to hdfs, then run the hdfs dfs command.  This wil create your personal directory in /user and make you the owner.  Now, when you run hdfs commands as devuser, you will have permission to modify objects in that folder.

 

CAREFUL: elevating to hdfs is like becoming root in linux.  You can do a lot of damage if you make a mistake.

 

 

Explorer

Hi @hughatoclc  thanks for this. this is a big help. however, how can I make my user also in hdfs supergroup? so that I can freely explore the hdfs and also can I run jobs using this as well?

@Mondi 

just update dfs.cluster.administrators with the admin username you want in hdfs config (and restart hdfs,yarn,MR2,... services)

ex: dfs.cluster.administrators = hdfs,ops

 

you can also use a HDFS administrators group (only one administrator group) using dfs.permissions.superusergroup

ex : dfs.permissions.superusergroup = operations

 

to verify config has been updated, once services restarted

hdfs getconf -confKey dfs.cluster.administrators

or

hdfs getconf -confKey dfs.permissions.superusergroup

 
; ;