Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How many YARN jobs is too many?

How many YARN jobs is too many?

New Contributor

Our cluster is currently struggling with the workload placed upon it. We are facing tens of thousands of jobs launched in a small time period. We've tuned YARN heavily and are now facing Oozie problems.

The question this raises is, what is an appropriate number of jobs to run on a cluster? Is there a best practice based on YARN capacity or the number of datanodes? I've done a lot of research and haven't found anything significant.

Thanks for your time

Don't have an account?
Coming from Hortonworks? Activate your account here