CDE Spark UI fails to load

@Craig-C Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDE expert @Azhar_Shaikh1  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

hi @Craig-C,

analyze the logs of the CDE pods, monitor the logs during the execution of the job and after the job ends, you will possibly find the problem.

@Craig-C Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.

Hello @Craig-C 

Greetings from Cloudera Community. Based on the Post, Spark UI is available for Running Jobs yet the Spark UI isn't available for Completed Jobs. 

Note that Spark UI for Running Jobs is Serviced via Driver yet the Spark UI for Completed Jobs is Serviced via Spark-History-Server. We have observed few Scenarios wherein Spark UI isn't available for Completed Jobs during High-Concurrency & have a Solution until a Long-Term Fix at CDE.

The Solution is a bit Verbose & as such, We shall recommend your Team to file a Support Case with Cloudera Support & We shall help your Team confirming the Issue reported above while applying the Mitigation Solution until a Long-Term Fix. 

Regards, Smarak

Hello ​@Craig-C 

I would like to report that the Spark UI is still not available for stopped jobs, even with Cloudera Data Engineering (CDE) version 1.27.0-b693.

We don't know if there will be any developments about it from 2027 onwards.

Regards,

Mario Sabetta