Logs from spark executors

[#1]

Where can I find the logs from spark functions? I can find the logs of the spark application, but if I need to debug a map function that I wrote, I cannot find these logs.

thanks!

[#2]

If you print or log to stdout, it goes to the stdout of the executor process, wherever that is running. In YARN-based deployment, you can use "yarn logs ..." to find the executor logs, I believe. Or dig through from the resource manager and find the executor process and its logs from the UI.

[#3]

Thanks for your reply.

I found in the UI the path to /var/log/spark/spark-worker. That is the spark executor's own log. I am actually looking for the logs that I added to functions that I wrote to map RDDs.

Please forgive me if I am using the wrong terminology here.