Member since
09-22-2014
5
Posts
1
Kudos Received
2
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
9416 | 07-15-2015 05:50 AM | |
17829 | 01-21-2015 05:29 AM |
08-11-2016
01:45 AM
@sgiri - This has already been answered above. If you use un-secure YARN, all your containers and the commands the container runs (such as in shell action) will run as the "yarn" user (i.e. the user your NodeManager daemon runs as). If you want to run containers in YARN as the actual submitting user, use the LinuxContainerExecutor instead. The caveat of using it is that you need your user accounts available on all NodeManagers so a setuid can be done against them when spawning the container processes.
... View more
07-15-2015
06:32 PM
Actually problem was in very agressive caching and overfilling spark.yarn.executor.memoryOverhead buffer and as cosequence OOM error. i just increase it and everything works now
... View more
02-04-2015
05:32 AM
Thanks for the workaround, it did work for the past week in the meantime. I just upgraded to CDH 5.3.1 and indeed it successfuly fixed the issue for me, so we do not need for this workaround anymore. Thanks again for the quick fix. Regards,
... View more