question Re: Adding a new user to the cluster in Archives of Support Questions (Read Only)

Adding a new user to the cluster

arunpoy — Mon, 08 Feb 2016 18:34:48 GMT

Say users are allowed to access a cluster from the edge node of a cluster. If the user wants to run jobs on the cluster, does the user should have his account on all the nodes of the cluster or just having an account on the edge node is enough?

Re: Adding a new user to the cluster

sshimpi — Tue, 09 Feb 2016 10:30:25 GMT

Hi @ARUNKUMAR RAMASAMY

No. User should not have account on all the nodes of the cluster. He should only have account on edge node.

For a new user there are 2 types are directories we need to create before the user access the cluster. 1- User home directory [directory created on Linux Filesystem ie. /home/<username>] 2- User HDFS directory [directory created on HDFS filesystem ie. /user/<username>]

As per neeraj, you only need to create HDFS home directory[ie. /user/<username>] on edge node. You can still run jobs with the new user on cluster, even if you havent created his home directory in linux.

==============

Below are 2 scenarios -

a. I added new user on edge node using command - #useradd <username> Before launching job on cluster, i need to create hdfs directory for user #sudo -u hdfs hadoop fs -mkdir </user/{username}> #sudo -u hdfs hadoop fs chown -R <username>:<grp_name> </user/{username}>

b. If the user is coming from ldap server, then you only need to make your edge node as ldap client and create a directory in HDFS using below command -

#sudo -u hdfs hadoop fs -mkdir </user/{username}> #sudo -u hdfs hadoop fs chown -R <username>:<grp_name> </user/{username}>

Let me know if this clears, what you are looking for.

Re: Adding a new user to the cluster

arunpoy — Tue, 09 Feb 2016 14:35:25 GMT

Thanks @Sagar Shimpi, i got a clear picture. one more question, what permission do you give for the root directory / on hdfs

Re: Adding a new user to the cluster

sshimpi — Tue, 09 Feb 2016 15:01:45 GMT

@ARUNKUMAR RAMASAMY

The root directory "/" permissions are 755[ie. rwxr-xr-x], by default these permissions are as per linux standards [ie umask].Umask for hdfs user is "022" And the owner and group are set to hdfs:hdfs.

Re: Adding a new user to the cluster

Vikas1 — Tue, 16 Feb 2016 16:59:48 GMT

This is right in beginner setup, when your hadoop cluster integrated with Kerberos security then authenticated user must exist in the every node where the task runs.

Re: Adding a new user to the cluster

sshimpi — Wed, 17 Feb 2016 12:26:40 GMT

Hi @Vikas Gadade - I think this is not the case. Even if you have kerberized cluster you still have only user added on Gateway/Client node. Make sure you have proper keyabs in place.

Hadoop Service always use Delegation token to nodes and access/execute jobs within kerberized cluster where it executes task.

Re: Adding a new user to the cluster

arunpoy — Wed, 17 Feb 2016 12:51:26 GMT

Hi @Sagar Shimpi, @Vikas Gadade may be correct. This is the info i got from one of the hadoop admins.

Can someone clarify or validate

In non-security mode (without kerberos), an account on edgenode is sufficient because user's container will run on slave nodes under the yarn account. In secure mode (with kerberos), you should have accounts on all machines, because user's container should run on slave nodes under the real username.

Re: Adding a new user to the cluster

Vikas1 — Wed, 17 Feb 2016 15:30:24 GMT

hi @ARUNKUMAR RAMASAMY, @Sagar Shimpi I verified with removing user "vgadade" from datanode, please find below output on secure hadoop cluster

Here is sample job output..

16/02/17 01:47:24 INFO mapreduce.Job: Job job_1455179809801_0007 running in uber mode : false

16/02/17 01:47:24 INFO mapreduce.Job: map 0% reduce 0%

16/02/17 01:47:26 INFO mapreduce.Job: Task Id : attempt_1455179809801_0007_m_000001_0, Status : FAILED

Application application_1455179809801_0007 initialization failed (exitCode=255) with output: main : command provided 0

main : run as user is vgadade

main : requested yarn user is vgadade

User vgadade not found

16/02/17 01:47:29 INFO mapreduce.Job: Task Id : attempt_1455179809801_0007_m_000001_1, Status : FAILED

Application application_1455179809801_0007 initialization failed (exitCode=255) with output: main : command provided 0

main : run as user is vgadade

main : requested yarn user is vgadade

User vgadade not found

16/02/17 01:47:30 INFO mapreduce.Job: map 50% reduce 0%

16/02/17 01:47:32 INFO mapreduce.Job: Task Id : attempt_1455179809801_0007_m_000001_2, Status : FAILED

Application application_1455179809801_0007 initialization failed (exitCode=255) with output: main : command provided 0

main : run as user is vgadade

main : requested yarn user is vgadade

User vgadade not found

16/02/17 01:47:37 INFO mapreduce.Job: map 100% reduce 0%

16/02/17 01:47:43 INFO mapreduce.Job: map 100% reduce 100%

16/02/17 01:47:44 INFO mapreduce.Job: Job job_1455179809801_0007 completed successfully

Job Counters

Failed map tasks=3

Launched map tasks=5

Launched reduce tasks=1

Other local map tasks=3

Data-local map tasks=1

Rack-local map tasks=1

Job Finished in 28.409 seconds

Nodemanager log out file ouput

2016-02-17 01:47:24,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container container_e21_1455179809801_0007_01_000002 startLocalizer is : 255

java.io.IOException: Application application_1455179809801_0007 initialization failed (exitCode=255) with output: main : command provided 0

main : run as user is vgadade

main : requested yarn user is vgadade

User vgadade not found

at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.startLocalizer(LinuxContainerExecutor.java:262)

2016-02-17 01:47:24,946 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_e21_1455179809801_0007_01_000002 transitioned from LOCALIZING to LOCALIZATION_FAILED

vgadade user added again on datanode and executed job ( container is successfully launched on previous Data node)

Here is sample job output..

16/02/17 01:55:46 INFO mapreduce.Job: Running job: job_1455179809801_0008

16/02/17 01:55:52 INFO mapreduce.Job: Job job_1455179809801_0008 running in uber mode : false

16/02/17 01:55:52 INFO mapreduce.Job: map 0% reduce 0%

16/02/17 01:55:58 INFO mapreduce.Job: map 100% reduce 0%

16/02/17 01:56:04 INFO mapreduce.Job: map 100% reduce 100%

16/02/17 01:56:04 INFO mapreduce.Job: Job job_1455179809801_0008 completed successfully

16/02/17 01:56:05 INFO mapreduce.Job: Counters: 50

Job Counters

Launched map tasks=2

Launched reduce tasks=1

Data-local map tasks=1

Rack-local map tasks=1

Job Finished in 20.333 seconds

Re: Adding a new user to the cluster

arunpoy — Thu, 18 Feb 2016 13:14:23 GMT

Thanks @Vikas Gadade. It helped.

Re: Adding a new user to the cluster

jstraub — Thu, 18 Feb 2016 14:53:36 GMT

@Sagar Shimpi @ARUNKUMAR RAMASAMY I agree with @Vikas Gadade, if you want to execute jobs with your user account, you have to make sure the user is available on every Nodemanager node!

Please see this => "YARN containers in a secure cluster use the operating system facilities to offer execution isolation for containers. Secure containers execute under the credentials of the job user. The operating system enforces access restriction for the container. The container must run as the user that submitted the application." more info => https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/SecureContainer.html

Re: Adding a new user to the cluster

jstraub — Thu, 18 Feb 2016 15:00:21 GMT

please see my comment above.

In secure mode you need local user accounts on all Nodemanager nodes

Re: Adding a new user to the cluster

frank93 — Wed, 28 Jun 2017 13:53:47 GMT

@Jonas Straub @Vikas Gadade

Thank you for clarify. Is there any workaround for that? Or is it fixed in HDP2.6? I also use SSSD + Kerberos only on management nodes. On Nodemanager hosts AD users does not exists thus YARN is not working.