Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Location of Job history logs in cloudera data science workbench

avatar
Explorer

Hi, I need the path where the job history logs are saved in cloudera data science workbench.

1 ACCEPTED SOLUTION

avatar
Expert Contributor

The console output is not stored anywhere besides in the CDSW Web UI itself.

 

If the user is executing YARN applications (Spark/Pyspark) then you would have the logs from any YARN containers stored as they normally would be and you could access them via the JobHistory Server Web UI.  Any Spark application run in CDSW will be in `--deploy-mode client` which means that the Spark Driver output is in the Web UI only - you have to manually copy it out of there and save in a file for future reference.

 

Nick

View solution in original post

1 REPLY 1

avatar
Expert Contributor

The console output is not stored anywhere besides in the CDSW Web UI itself.

 

If the user is executing YARN applications (Spark/Pyspark) then you would have the logs from any YARN containers stored as they normally would be and you could access them via the JobHistory Server Web UI.  Any Spark application run in CDSW will be in `--deploy-mode client` which means that the Spark Driver output is in the Web UI only - you have to manually copy it out of there and save in a file for future reference.

 

Nick