Spark Application history not found Application is still in progress

petibonvm — Fri, 16 Sep 2022 09:39:13 GMT

Hi,

Since i performed the cloudera upgrade from CDH 5.3.2 to 5.4.5, i have an issue with the spark history server.

I found a similar problem on https://github.com/apache/spark/pull/4848

When application is finished running abnormally (Ctrl + c for example), the history event log file is still ends with .inprogress suffix. And the application state can not be showed on webUI, User can only see "Application history not foud xxxx, Application xxx is still in progress".

Any idea ?

regards,

Re: Spark Application history not found Application is still in progress

srowen — Wed, 26 Aug 2015 20:07:44 GMT

This may be too unspecific to be helpful, but I recall several JIRAs fixed for Spark 1.4 that concern the .inprogress files and history server. I expect that whatever this is could be related. If so, then the fix would be coming in 5.5 at the latest.

question Re: Spark Application history not found Application is still in progress in Archives of Support Questions (Read Only)

Spark Application history not found Application is still in progress

Re: Spark Application history not found Application is still in progress