CDSW Jobs are shown as "Success" when python code throws an Exception

[#1]

We have several CDSW jobs set up where we call a python script with a main() function which is called via

if __name__ ==  "__main__":

    main()

Our problem is that these jobs are erronously shown as "Success" even if the python code throws an Exception.

How can we set up our CDSW such that these jobs are correctly shown as "Failure"?

[#2]

@cdsw_user_23432 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our CDSW experts @Gopinath and @Mike  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

[#3]

Hi @DianaTorres , any updates on this?

[#4]

Thank you @DianaTorres .

I'm looking forward to hearing back from your team

[#5]

seems to be a gap in the exception handling.  Could you please share the actual script snippet for analysis.

[#6 · Accepted solution]

Thank you for your reply, which made me take a closer look at which exceptions were handled unexpectedly and I found that the use of rich.traceback.install() in one of the scripts messed up the return status of the CDSW Job.

The issue was solved by removing rich.traceback.install() from the code, which is only cosmetic anyhow

[#7]

Congratulations on resolving the issue @cdsw_user_23432. Please use the "Accept as Solution" button to mark the reply that resolved the issue for you. It helps others in a similar situation find assistance in the future.