Cloudera Community

Support Questions

Find answers, ask questions, and share your expertise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Advanced Search

Why do "spark-shell and spark-submit" keep showing WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!

avatar
author-rank JuliaC
Explorer

Created on ‎05-24-2021 08:09 AM - last edited on ‎05-24-2021 10:57 PM by Community Manager Community Manager

I am new to Cloudera CDP 7.1.4.
I could run spark-shell and spark-submit with no problem even they kept showing this WARN.
How do I stop the Spark job's requesting executors before the Application Master has registered?
Should I tune some properties from CM-Home> Cluster 1> Yarn> Configuration OR edit conf files in RM: /var/run/cloudera-scm-agent/process/LATEST-NUMBER-yarn-RESOURCEMANAGER?

 

My cluster:

Yarn> Scheduler Class: CapacityScheduler
Yarn> Queue Manager Service: YARN Queue Manager

 

Thanks.

1,418 Views
0 Kudos
1 REPLY 1

avatar
author-rank JuliaC
Explorer

Created on ‎05-24-2021 08:24 AM - edited ‎05-24-2021 08:30 AM

My cluster:

Yarn> Scheduler Class: CapacityScheduler
Yarn> Queue Manager Service: YARN Queue Manager

1,404 Views
0 Kudos