Support Questions
Find answers, ask questions, and share your expertise
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Pyspark logs to different file


Pyspark logs to different file


Hi All,

I want logger=INFO to be stored in a different file.

I am running pyspark script using JupyterHub so I want to know where all i need to change the configurations to achieve the activity of getting logs into a different file path of my wish. I mean do i need to change the jupyterhub pyspark config files.

I found this link :

But it is for spark i need the same for pyspark. If i update spark does it work for pySpark as well?

Please help in this.


Manjunath P N

Don't have an account?
Coming from Hortonworks? Activate your account here