I am trying to run a spark application that runs fine in local mode.
I am running like this:
/usr/hdp/18.104.22.168-205/spark2/bin/spark-submit --class MyMain \ --master yarn \ --deploy-mode cluster \ --executor-memory 2G \ --num-executors 10 \ framework-1.0.0-0-all.jar
But it takes forever to start and in hadoop application UI I see this status:
YarnApplicationState:ACCEPTED: waiting for AM container to be allocated, launched and register with RM
In the console I see this line every second for over 10 minutes:
17/10/24 15:57:30 INFO Client: Application report for application_1508848914801_0003 (state: ACCEPTED)
That didn't help. I now have 3 node managers. still it seems stuck in Accepted state. Note: under: http:/my-node:8042/node/allApplications I do see the container running and logs show the application IS running. So - even more strange that the app seems to stuck in Accepted mode for so long...
Can you please check the resource manager UI to see the actual reason. Login to RM UI -> Applications -> Accepted -> Click on app and check for the diagnostics field. Please paste the output here