Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Adding a new user to the cluster

avatar

Say users are allowed to access a cluster from the edge node of a cluster. If the user wants to run jobs on the cluster, does the user should have his account on all the nodes of the cluster or just having an account on the edge node is enough?

1 ACCEPTED SOLUTION

avatar
Super Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
11 REPLIES 11

avatar
Super Guru
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar

Thanks @Sagar Shimpi, i got a clear picture. one more question, what permission do you give for the root directory / on hdfs

avatar
Super Guru

@ARUNKUMAR RAMASAMY

The root directory "/" permissions are 755[ie. rwxr-xr-x], by default these permissions are as per linux standards [ie umask].Umask for hdfs user is "022" And the owner and group are set to hdfs:hdfs.

avatar
Rising Star

This is right in beginner setup, when your hadoop cluster integrated with Kerberos security then authenticated user must exist in the every node where the task runs.

avatar
Super Guru

Hi @Vikas Gadade - I think this is not the case. Even if you have kerberized cluster you still have only user added on Gateway/Client node. Make sure you have proper keyabs in place.

Hadoop Service always use Delegation token to nodes and access/execute jobs within kerberized cluster where it executes task.

avatar

@Sagar Shimpi @ARUNKUMAR RAMASAMY I agree with @Vikas Gadade, if you want to execute jobs with your user account, you have to make sure the user is available on every Nodemanager node!

Please see this => "YARN containers in a secure cluster use the operating system facilities to offer execution isolation for containers. Secure containers execute under the credentials of the job user. The operating system enforces access restriction for the container. The container must run as the user that submitted the application." more info => https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/SecureContainer.html

avatar

Hi @Sagar Shimpi, @Vikas Gadade may be correct. This is the info i got from one of the hadoop admins.

Can someone clarify or validate

In non-security mode (without kerberos), an account on edgenode is sufficient because user's container will run on slave nodes under the yarn account. In secure mode (with kerberos), you should have accounts on all machines, because user's container should run on slave nodes under the real username.

avatar

please see my comment above.

In secure mode you need local user accounts on all Nodemanager nodes

avatar
Super Collaborator

@Jonas Straub @Vikas Gadade

Thank you for clarify. Is there any workaround for that? Or is it fixed in HDP2.6? I also use SSSD + Kerberos only on management nodes. On Nodemanager hosts AD users does not exists thus YARN is not working.