Support Questions

Find answers, ask questions, and share your expertise

Spark Log4j - Driver and executor log in console

New Contributor

Hello Team,

 

I am running spark job in cluster mode using java, the jobs are running without issue but not able to see the logs through console both driver and executor log,

 

but spark UI able to see the executor logs.

 

I want to see the driver and executor logs in console while running the spark job. Could you please assist someone to help this issue asap.

 

1 REPLY 1

Expert Contributor

Hi @Sugumar 

 

As we discussed in https://community.cloudera.com/t5/Support-Questions/Java-Spark-driver-and-executor-logs-in-cluster-m... post, if your cluster is DSE then please check DSE team. 

If it is yarn then u can use following command:

yarn logs -applicationId <Application_ID> > application_id.log
Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.