- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Why do "spark-shell and spark-submit" keep showing WARN cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
Created on
‎05-24-2021
08:09 AM
- last edited on
‎05-24-2021
10:57 PM
by
VidyaSargur
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am new to Cloudera CDP 7.1.4.
I could run spark-shell and spark-submit with no problem even they kept showing this WARN.
How do I stop the Spark job's requesting executors before the Application Master has registered?
Should I tune some properties from CM-Home> Cluster 1> Yarn> Configuration OR edit conf files in RM: /var/run/cloudera-scm-agent/process/LATEST-NUMBER-yarn-RESOURCEMANAGER?
My cluster:
Yarn> Scheduler Class: CapacityScheduler
Yarn> Queue Manager Service: YARN Queue Manager
Thanks.
Created on ‎05-24-2021 08:24 AM - edited ‎05-24-2021 08:30 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My cluster:
Yarn> Scheduler Class: CapacityScheduler
Yarn> Queue Manager Service: YARN Queue Manager
