couldn't able to execute spark jar in cluster mode

[#1]

I am working on Spark with scala , I have created a jar and tried to execute in the cluster mode as below spark-submit --class com.test.sales.LoadSalesData \ --master yarn \ --deploy-mode cluster \ --driver-memory 4g \

--executor-memory 2g \

--executor-cores 1 \

test_salesdata_2.10-1.0.jar

but I am getting below error ,

17/01/18 05:15:16 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:17 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:18 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:19 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:20 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:21 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:22 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:23 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED)

client token: N/A diagnostics: Application application_1483728394968_0141 failed 2 times due to AM Container for appattempt_1483728394968_0141_000002 exited with exitCode: 1 3

i could able to execute the same code in spark-shell with out any issues.. could you please let me know ,if I am missing anything to run the spark job in cluster mode. thank you !

[#2]

Looks like your job was submitted to Yarn. Can you log information from spark history UI or from Yarn

Regards

Pranay Vyas

[#3]

Please check from the Yarn RM UI why the application master failed? The AM logs would give you indication depending on exceptions as to what caused the failures . Once that is identified, you can proceed to fix it.

[#4 · Accepted solution]

@Satya

yarn application log could give the actual error.

Can you also check if you have set master inside the code?

[#5]

Thank you.. It has been resolved now... i forgot to remove .setMaster("local") in the spark jar.. after removing that job went fine..

[#6]

Please accept the answer by Sandeep so that this question can be marked as addressed.