Reply
Expert Contributor
Posts: 64
Registered: ‎11-04-2016

Re: YARN force nobody user on all jobs (and so they fail)

[ Edited ]

Hi @Harsh J

 

Thanks for your response. Yes, these are the two configs you mentioned, and also I checked all the "safety valves" there is nothing related to any Linux or cgroups:

 

image.png

image.png

 

I have even remove the "nobody" user from allowed and left the "nonsecure-mode.lcaol-user" empty, but still says "nobody". If I revert all the changes, it says "yarn". So these configs impact something somewhere.

 

Cloudera Express: 5.15.1

Java Version: 1.8.0_181

CDH: 5.15.1-1.cdh5.15.1.p0.4

 

 

UPDATE: one more thing that might be useful, when I download client configuration from CM, I can't find these two configs in any of the configs. Not sure if that is normal.

 

Best,

Maziyar

Expert Contributor
Posts: 64
Registered: ‎11-04-2016

Re: YARN force nobody user on all jobs (and so they fail)

Hi @Harsh J

 

You mentioning the Safe Valve gave me an idea! I thought maybe the UI in CM is not setting one or both of those key/values. So I did this manually and it worked! Now every container asked by Spark Pipe() has the same owner as the Spark application itself (no more nobody or yarn! - there must be something with the UI that won't map one of those two configs back to yarn-site.xml):

 

image.png

Announcements