Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can maximum task failures attempts kill the job?

Highlighted

Can maximum task failures attempts kill the job?

Expert Contributor

Hi all,

My job just failed (1 task failed with 4 attempts on a single node) and rest of the tasks got killed. The node on which the task failed just had a disk failure. My question is:

when a disk failure occurs namenode excludes that disk from accessing any data, so why did the task fail in 4 attempts with a second gap between the attempts on same node? and can this result in the whole job failure? how can i avoid this situation in the future? How can i make the second task attempt on a different node?

I am using hdp 2.5.3

Thank you..

Don't have an account?
Coming from Hortonworks? Activate your account here