Support Questions
Find answers, ask questions, and share your expertise

Spark jobs fail with "Received launch task command but executor was null"

Highlighted

Spark jobs fail with "Received launch task command but executor was null"

Hi,

Our spark jobs are failing with "Received launch task command but executor was null" error. As a work around, we increased the spark.yarn.maxexecutorfailures from 3 to 300 and the job went successful after several executor failures.

Could any one let us know the reasons behind this failure.

We are using Spark 1.6.3 in HDP2.4. We are observing these failures since we upgraded the RAM of our hadoop cluster servers.