I noticed the following submiting batch job (jar) in Yarn mode. I am able to submit a jar and seem it is being issued to spark-submit correctly (the logged spark-submit command runs fine without livy context). However, after it is done, the status of the batch is as below: {"id":0,"state":"running","log":["\t diagnostics: N/A","\t ApplicationMaster host: 10.0.0.167","\t ApplicationMaster RPC port: 0","\t queue: default","\t start time: 1443732089755","\t final status: SUCCEEDED","\t tracking URL: http://servername:8088/proxy/application_1443668789912_0010/","\t user: root","15/10/01 20:42:31 INFO Utils: Shutdown hook called","15/10/01 20:42:31 INFO Utils: Deleting directory /tmp/spark-0343eaf9-c8e6-4f85-9200-6d3366851b1e"]} Livy still thinks the state is running but the log seems to indicate that the job is finished and SUCCEEDED. Any idea what might be happening here? Also once batch is in that state and I issue a delete, it will hang for a bit and then give the following timeout response. < HTTP/1.1 504 Gateway Timeout < Date: Thu, 01 Oct 2015 20:54:58 GMT < Content-Type: application/json; charset=UTF-8 < Content-Length: 15 < Server: Jetty(9.2.z-SNAPSHOT) < * Connection #0 to host headnode0 left intact * Closing connection #0 Gateway timeout Thanks Lin
... View more
