Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive : Spark job cancelled

Highlighted

Hive : Spark job cancelled

Contributor

Hi. I've a query running over Spark engine failing after x hours. There is no explicit message saying the reason.

 

17/03/29 23:26:38 INFO cluster.YarnClusterScheduler: Executor 91 on INVHDP03 killed by driver.
17/03/29 23:26:38 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 91 from BlockManagerMaster.
17/03/29 23:26:38 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(91, INVHDP03, 57485)
17/03/29 23:26:38 INFO storage.BlockManagerMaster: Removed 91 successfully in removeExecutor
17/03/29 23:26:38 INFO spark.ExecutorAllocationManager: Existing executor 91 has been removed (new total is 3)
17/03/30 07:54:25 INFO client.RemoteDriver: Shutting down remote driver.
17/03/30 07:54:25 INFO scheduler.DAGScheduler: Asked to cancel job 93
17/03/30 07:54:25 INFO client.RemoteDriver: Failed to run job 1f9aedd6-3c90-4392-ba91-e62f8df843a6
java.lang.InterruptedException
	at java.lang.Object.wait(Native Method)
	at java.lang.Object.wait(Object.java:503)
	at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:371)
	at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:335)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
17/03/30 07:54:25 INFO cluster.YarnClusterScheduler: Cancelling stage 135
17/03/30 07:54:25 INFO cluster.YarnClusterScheduler: Stage 135 was cancelled
17/03/30 07:54:25 INFO scheduler.DAGScheduler: ShuffleMapStage 135 (mapPartitionsToPair at ReduceTran.java:32) failed in 43240.482 s