Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Why do "spark-shell and spark-submit" keep showing WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!

avatar
Explorer

I am new to Cloudera CDP 7.1.4.
I could run spark-shell and spark-submit with no problem even they kept showing this WARN.
How do I stop the Spark job's requesting executors before the Application Master has registered?
Should I tune some properties from CM-Home> Cluster 1> Yarn> Configuration OR edit conf files in RM: /var/run/cloudera-scm-agent/process/LATEST-NUMBER-yarn-RESOURCEMANAGER?

 

My cluster:

Yarn> Scheduler Class: CapacityScheduler
Yarn> Queue Manager Service: YARN Queue Manager

 

Thanks.

1 REPLY 1

avatar
Explorer

My cluster:

Yarn> Scheduler Class: CapacityScheduler
Yarn> Queue Manager Service: YARN Queue Manager