Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark jobs fail with "Received launch task command but executor was null"

Highlighted

Spark jobs fail with "Received launch task command but executor was null"

Hi,

Our spark jobs are failing with "Received launch task command but executor was null" error. As a work around, we increased the spark.yarn.maxexecutorfailures from 3 to 300 and the job went successful after several executor failures.

Could any one let us know the reasons behind this failure.

We are using Spark 1.6.3 in HDP2.4. We are observing these failures since we upgraded the RAM of our hadoop cluster servers.

Don't have an account?
Coming from Hortonworks? Activate your account here