Hive fails with ExitCodeException exitCode=1:

[#1]

Hi,

I am trying to run a simple hive query, and it keeps failing with following error. To make it simple, I have disabled Authentication/Authorization for Hive. Still the same error. It runs as user 'nobody'

Container exited with a non-zero exit code 1

]], TaskAttempt 3 failed, info=[Container container_e12_1480595328764_0023_01_000005 finished with diagnostics set to [Container failed, exitCode=1. Exception from container-launch.

Container id: container_e12_1480595328764_0023_01_000005

Exit code: 1

Stack trace: ExitCodeException exitCode=1:

at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)

at org.apache.hadoop.util.Shell.run(Shell.java:456)

at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)

at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:367)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)

at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Shell output: main : command provided 1

main : run as user is nobody

main : requested yarn user is hdpuser001

Thanks,

[#2 · Accepted solution]

@hello hadoop

To enable LaunchContainerExecutor to impersonate the actual user, you need to change following property to false.

yarn.nodemanager.linux-container-executor.nonsecure-mode.limit-users

and when you do this, then user "hdpuser001" must exist on all nodes in the cluster, otherwise, part of the job that's running on a node without this user will fail.

[#3]

Thank you @mqureshi

I have set the flag to false and used 'hive' user instead of 'hdpuser001'. So it is now reflecting "run as user is hive". However, the Hive query is still failing when the engine is set to Tez. It works perfectly fine with MR. There are not much logs available as well

[#4]

@hello hadoop

Logs must be there. Without logs, we can't help. Can you please check log files under /var/log/hive?

[#5]

@mqureshi - Thank you. I figured out that the memory settings were wrong for Tez. I fixed it and It works perfectly fine. The following link was helpful as well - http://www.hadoopadmin.co.in/hive/tez-job-fails-with-vertex-failure-error/