Reply
New Contributor
Posts: 3
Registered: ‎05-07-2018

Location of Job history logs in cloudera data science workbench

Hi, I need the path where the job history logs are saved in cloudera data science workbench.

Highlighted
Cloudera Employee
Posts: 33
Registered: ‎06-10-2015

Re: Location of Job history logs in cloudera data science workbench

The console output is not stored anywhere besides in the CDSW Web UI itself.

 

If the user is executing YARN applications (Spark/Pyspark) then you would have the logs from any YARN containers stored as they normally would be and you could access them via the JobHistory Server Web UI.  Any Spark application run in CDSW will be in `--deploy-mode client` which means that the Spark Driver output is in the Web UI only - you have to manually copy it out of there and save in a file for future reference.

 

Nick

Announcements