Support Questions

Find answers, ask questions, and share your expertise

Hive fails with ExitCodeException exitCode=1:

Hi,

I am trying to run a simple hive query, and it keeps failing with following error. To make it simple, I have disabled Authentication/Authorization for Hive. Still the same error. It runs as user 'nobody'

Container exited with a non-zero exit code 1

]], TaskAttempt 3 failed, info=[Container container_e12_1480595328764_0023_01_000005 finished with diagnostics set to [Container failed, exitCode=1. Exception from container-launch.

Container id: container_e12_1480595328764_0023_01_000005

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)

at org.apache.hadoop.util.Shell.run(Shell.java:456)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)

at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:367)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Shell output: main : command provided 1

main : run as user is nobody

main : requested yarn user is hdpuser001

Thanks,

1 ACCEPTED SOLUTION

Super Guru
@hello hadoop

To enable LaunchContainerExecutor to impersonate the actual user, you need to change following property to false.

yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users

and when you do this, then user "hdpuser001" must exist on all nodes in the cluster, otherwise, part of the job that's running on a node without this user will fail.

View solution in original post

4 REPLIES 4

Super Guru
@hello hadoop

To enable LaunchContainerExecutor to impersonate the actual user, you need to change following property to false.

yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users

and when you do this, then user "hdpuser001" must exist on all nodes in the cluster, otherwise, part of the job that's running on a node without this user will fail.

Thank you @mqureshi

I have set the flag to false and used 'hive' user instead of 'hdpuser001'. So it is now reflecting "run as user is hive". However, the Hive query is still failing when the engine is set to Tez. It works perfectly fine with MR. There are not much logs available as well

Super Guru

@hello hadoop

Logs must be there. Without logs, we can't help. Can you please check log files under /var/log/hive?

@mqureshi - Thank you. I figured out that the memory settings were wrong for Tez. I fixed it and It works perfectly fine. The following link was helpful as well - http://www.hadoopadmin.co.in/hive/tez-job-fails-with-vertex-failure-error/

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.