Member since
01-18-2017
2
Posts
0
Kudos Received
0
Solutions
01-26-2017
12:46 PM
Thank you.. It has been resolved now... i forgot to remove .setMaster("local") in the spark jar.. after removing that job went fine..
... View more
01-18-2017
12:08 PM
I am working on Spark with scala , I have created a jar and tried to execute in the cluster mode as below
spark-submit --class com.test.sales.LoadSalesData \
--master yarn \ --deploy-mode cluster \
--driver-memory 4g \ --executor-memory 2g \ --executor-cores 1 \ test_salesdata_2.10-1.0.jar but I am getting below error , 17/01/18 05:15:16 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:17 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:18 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:19 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:20 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:21 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:22 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) 17/01/18 05:15:23 INFO Client: Application report for application_1483728394968_0141 (state: ACCEPTED) client token: N/A
diagnostics: Application application_1483728394968_0141 failed 2 times due to AM Container for appattempt_1483728394968_0141_000002 exited with exitCode: 1
3 i could able to execute the same code in spark-shell with out any issues.. could you please let me know ,if I am missing anything to run the spark job in cluster mode.
thank you !
... View more
Labels:
- Labels:
-
Apache Spark