Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

were i can set Max number of executor failures ?

avatar
Rising Star

can i set it via the ambari config?

2 REPLIES 2

avatar
Master Mentor

@Ilia K


You might be interested in the following two properties. Please check the below doc if you are looking out for mentioned properties?
https://spark.apache.org/docs/2.2.0/configuration.html

"spark.dynamicAllocation.enabled": Whether to use dynamic resource allocation, which scales the number of executors registered with this application up and down based on the workload. (default value: false)

"spark.dynamicAllocation.maxExecutors":
Upper bound for the number of executors if dynamic allocation is enabled. (Default Value: infinity)



avatar
Rising Star

i don't think this helps me because i want to increase the failure limit to more than 20