Skip to main content
Craig-C
Craig-C
Bronze III
December 30, 2024
Question

CDE Spark UI fails to load

Hi

I've got a CDE 1.22.0-h1-b3 cluster that fails to load the Spark UI after the job has completed. I can see the Spark UI during the running of a job but it fails to load/open once the job has completed.

I see an http GET to the /jobs endpoint fails with a 404 Not Found.

Have you come across this error and resolved it please?

Cheers

Craig

    4 replies

    DianaTorres
    Community Manager
    DianaTorres
    Community Manager
    December 30, 2024

    @Craig-C Welcome to the Cloudera Community!

    To help you get the best possible solution, I have tagged our CDE expert @Azhar_Shaikh1  who may be able to assist you further.

    Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

    Translate
    Regards, Diana Torres, Senior Community Moderator
      yagoaparecidoti
      yagoaparecidoti
      Gold ⭐️
      January 8, 2025

      hi @Craig-C,

      analyze the logs of the CDE pods, monitor the logs during the execution of the job and after the job ends, you will possibly find the problem.

       

      Translate
      DianaTorres
      Community Manager
      DianaTorres
      Community Manager
      January 15, 2025

      @Craig-C Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.

      Translate
      Regards, Diana Torres, Senior Community Moderator
      smdas
      Gold ⭐️⭐️
      smdas
      Gold ⭐️⭐️
      April 1, 2025

      Hello @Craig-C 

      Greetings from Cloudera Community. Based on the Post, Spark UI is available for Running Jobs yet the Spark UI isn't available for Completed Jobs. 

      Note that Spark UI for Running Jobs is Serviced via Driver yet the Spark UI for Completed Jobs is Serviced via Spark-History-Server. We have observed few Scenarios wherein Spark UI isn't available for Completed Jobs during High-Concurrency & have a Solution until a Long-Term Fix at CDE.

      The Solution is a bit Verbose & as such, We shall recommend your Team to file a Support Case with Cloudera Support & We shall help your Team confirming the Issue reported above while applying the Mitigation Solution until a Long-Term Fix. 

      Regards, Smarak

      Translate
      Terms & ConditionsAccessibility statement