Created on 10-07-2014 11:20 PM - edited 09-16-2022 02:09 AM
When I execute the following in yarn-client mode its working fine and giving the result properly, but when i try to run in Yarn-cluster mode i am getting error
spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client /home/abc/spark/examples/lib/spark-examples_2.10-1.0.0-cdh5.1.0.jar 10
The above code works fine, but when i execute the same code in yarn cluster mode i amgetting the following error.
14/10/07 09:40:24 INFO Client: Application report from ASM: application identifier: application_1412117173893_1150 appId: 1150 clientToAMToken: Token { kind: YARN_CLIENT_TOKEN, service: } appDiagnostics: appMasterHost: N/A appQueue: root.default appMasterRpcPort: -1 appStartTime: 1412689195537 yarnAppState: ACCEPTED distributedFinalState: UNDEFINED appTrackingUrl: http://spark.abcd.com:8088/proxy/application_1412117173893_1150/ appUser: abc 14/10/07 09:40:25 INFO Client: Application report from ASM: application identifier: application_1412117173893_1150 appId: 1150 clientToAMToken: null appDiagnostics: Application application_1412117173893_1150 failed 2 times due to AM Container for appattempt_1412117173893_1150_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:511) at org.apache.hadoop.util.Shell.run(Shell.java:424) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:656) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:279) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) main : command provided 1 main : user is abc main : requested yarn user is abc Container exited with a non-zero exit code 1 .Failing this attempt.. Failing the application. appMasterHost: N/A appQueue: root.default appMasterRpcPort: -1 appStartTime: 1412689195537 yarnAppState: FAILED distributedFinalState: FAILED appTrackingUrl: spark.abcd.com:8088/cluster/app/application_1412117173893_1150 appUser: abc
Where may be the problem? sometimes when i try to execute in yarn-cluster mode i am getting the following , but i dint see any result
14/10/08 01:51:57 INFO Client: Application report from ASM: application identifier: application_1412117173893_1442 appId: 1442 clientToAMToken: Token { kind: YARN_CLIENT_TOKEN, service: } appDiagnostics: appMasterHost: spark.abcd.com appQueue: root.default appMasterRpcPort: 0 appStartTime: 1412747485673 yarnAppState: FINISHED distributedFinalState: SUCCEEDED appTrackingUrl: http://spark.abcd.com:8088/proxy/application_1412117173893_1442/A appUser: abc
Thanks
Created 11-24-2014 12:17 AM
I met the same issue now. Have you find out the way to solve it?
Created 12-23-2014 01:48 AM
try the below command to get the detailed log.
$HADOOP_HOME/bin/yarn logs -applicationId application_1419229907721_0010
Created 06-18-2015 12:54 AM
You are not specifying the jar that contains that class (the examples jar).
It could be that the jar is included automatically in local mode but not in the yarn classpath.
Have a loot at the nodemanager log for the node that tried to run it to veryfy if it's a classpath issue
Created 01-29-2016 07:35 PM
Created 01-29-2016 10:35 PM
An exit code -1 means Java crashed, normally that is due to classpath or memory settings
Created 06-12-2016 08:09 AM
How was this fixed?
Created 06-13-2016 04:42 AM
For the application ownload the application logs and check what the error is in the logs:
yarn application -applicationId APPID -appOwner USERID
Check the exit codes of the application and you should be able to tell in a bit more detail what is going on.
Wilfred
Created 06-19-2015 03:24 AM
This error:
main : command provided 1 main : user is abc main : requested yarn user is abc Container exited with a non-zero exit code 1
Looks like the exit code from the linux container executor.
In cluster mode the driver runs inside the same container as the Application Master which makes a difference.
As other people have said already get the logs from the containers by running:
yarn logs -applicationId APPID
Make sure that you run it as the user "abc" (same as the user that executes the spark command).
Wilfred
I am running my spark streaming application using spark-submit on yarn-cluster. When I run it on local mode it is working fine. But when I try to run it on yarn-cluster using spark-submit, it runs for some time and then exits with following execption.
Diagnostics:Exception from container-launch.Container id: container_1435576266959_1208_02_000002Exit code:13Stack trace:ExitCodeException exitCode=13: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
Any help will be appreciated.