Running as root is not allowed

[#1]

I tried to set up Impala to use YARN resource management. This requires (except from other things) to turn on Linux Container Execution (LCE) on all hots and 

configuring YARN to use LCE.

The problem is, that when I tried to run a Spark job under root account YARN refused to do ths. First, it was an error message about nobody user.

Since YARN is by default configured to use this user, I changed the yarn.nodemanager.linux-container-executor.nonsecure-mode.local.user to false 

in safety valve for yarn-site.xml.

Regarding the documentation this should enforce that every action in the container is executed under the user who submitted the job.

I tried to add root to the whitelist of allowed users in YARN (allowed.system.users) and setting min.user.id to 1, but nothng helped.

Yarn still is refusing to start a job under root.

Any ideas?

thanks

Tomas

[#2 · Accepted solution]

Hi Tomas,

Root has uid=0, that's why.
It is highly recommended not to use root to run jobs for various reasons: security, stability, portability etc.
The best solution is to run container as standard user, which needs certain level of permissions. Here is example with ubuntu:
https://docs.docker.com/installation/ubuntulinux/#giving-non-root-access

Please let us know if it worked.

Regards,
Pawel Niemiec

[#3]

Hello,

           yarn makes three checks ( source code )  :

1. compare the name of the user with string root with string compare (strcmp(user, "root") == 0
2. verify if your user is white listed   ( !is_whitelisted(user)
3. check the uid of the user with minuid.  ( user_info->pw_uid < min_uid  )

For now the only workaround I found is to create a new user with UID and GID equal to 0 and insert the name of the user in white listed and set min user id to 0.

There is an important motivation to use root: if you need to use distcp on a target location that is an NFS filesystem or a sharable filesystem mounted local on the datanode/workernode to make a backup.

Infact in that case, if you run a job with a normal user, it's not possible to change the owner of the file, so the distcp backup will fails.  Obviously if you run as root it will fail too for the hard coded control.

Kind Regards