Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark Log4j - Driver and executor log in console

avatar
New Contributor

Hello Team,

 

I am running spark job in cluster mode using java, the jobs are running without issue but not able to see the logs through console both driver and executor log,

 

but spark UI able to see the executor logs.

 

I want to see the driver and executor logs in console while running the spark job. Could you please assist someone to help this issue asap.

 

1 REPLY 1

avatar
Master Collaborator

Hi @Sugumar 

 

As we discussed in https://community.cloudera.com/t5/Support-Questions/Java-Spark-driver-and-executor-logs-in-cluster-m... post, if your cluster is DSE then please check DSE team. 

If it is yarn then u can use following command:

yarn logs -applicationId <Application_ID> > application_id.log