Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Location of Job history logs in cloudera data science workbench

Solved Go to solution
Highlighted

Location of Job history logs in cloudera data science workbench

New Contributor

Hi, I need the path where the job history logs are saved in cloudera data science workbench.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Location of Job history logs in cloudera data science workbench

Cloudera Employee

The console output is not stored anywhere besides in the CDSW Web UI itself.

 

If the user is executing YARN applications (Spark/Pyspark) then you would have the logs from any YARN containers stored as they normally would be and you could access them via the JobHistory Server Web UI.  Any Spark application run in CDSW will be in `--deploy-mode client` which means that the Spark Driver output is in the Web UI only - you have to manually copy it out of there and save in a file for future reference.

 

Nick

1 REPLY 1

Re: Location of Job history logs in cloudera data science workbench

Cloudera Employee

The console output is not stored anywhere besides in the CDSW Web UI itself.

 

If the user is executing YARN applications (Spark/Pyspark) then you would have the logs from any YARN containers stored as they normally would be and you could access them via the JobHistory Server Web UI.  Any Spark application run in CDSW will be in `--deploy-mode client` which means that the Spark Driver output is in the Web UI only - you have to manually copy it out of there and save in a file for future reference.

 

Nick

Don't have an account?
Coming from Hortonworks? Activate your account here