Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

MR App Container - Process User

MR App Container - Process User

I am working in a kerberized cluster where a user is submitting a SQOOP job on the edge node. The actual MR job on the worker nodes run under the user 'yarn'.

Is there any way we can configure YARN to use the end user's user id for launching the MR process on the worker nodes?

4 REPLIES 4

Re: MR App Container - Process User

Super Guru

@bsaini - is it only for sqoop job or any mapreduce job? e.g. wordcount/pi example.

2. by which user are you running job? is that user exists on other nodes in the cluster or only on edge node?

Re: MR App Container - Process User

@Kuldeep Kulkarni

It applies to both (MR and Sqoop) but the job in question is being kicked off using SQOOP. It is more critical for the jobs that are making the connection to external systems like SQL Server as this whole environment is kerberized and without the user's kerberos ticket (which will happen only if the job is running under that user's id)

Now as per this documentation - https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/SecureContainer.html ... "YARN containers in a secure cluster use the operating system facilities to offer execution isolation for containers. Secure containers execute under the credentials of the job user. The operating system enforces access restriction for the container. The container must run as the user that submitted the application."

Re: MR App Container - Process User

Super Guru

@bsaini

1. Do you have user xyz who is submitting job on all the nodemanagers?

2. Do you have below parameter set in /etc/hadoop/conf/container-executor.cfg

banned.users=hdfs,yarn,mapred

3. Can you confirm if you are using below property in yarn-site.xml

yarn.nodemanager.container-executor.class = org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor

Re: MR App Container - Process User

Super Guru

@bsaini - Any updates on this?

Don't have an account?
Coming from Hortonworks? Activate your account here