Error info: 16/04/24 13:42:24 WARN YarnScheduler: Initial job has not accepted any resources

[#1]

Hi,

In HDP sandbox 2.4,

When to start Spark with

./bin/spark-shell --master yarn --deploy-mode client

After create a job, there's error:

16/04/24 13:42:24 WARN YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

[#2]

@JR Cao

Hi,

It means that either your cluster doesn't have sufficient resources or in some cases non of node managers connected to the resource manager.

Please cross check your resource manager UI and look for "Memory used", "Memory Total" and "Active Nodes" section.

http://<RM IP>:8088

see attached screenshot.

[#3 · Accepted solution]

Resolved. Need to edit the spark-env configure file to enable spark yarn client mode.

[#4]

@JR Cao

Thanks for the update, I think you don't need to specify spark-env since you already had --deploy-mode client.