Support Questions

Find answers, ask questions, and share your expertise

How to view the log of submitted spark jar program?

avatar
Rising Star

I have submitted a spark java program via "Spark Submit Jar" and it looks running well.

However, when I click the logs link in specified application in Job tab in hue,  it shows "cannot acces: /jobbrowser/jobs/appliacation_****/single_logs."

 

So how can I find logs of running spark application? 

2 REPLIES 2

avatar
Champion

 

You can hit the below url for currently runining jobs for 

 http://spark_driver_host:4040

monitoring purpose 

 

To see the completed jobs you can see in Spark History Web UI 

 

Let me know if you need any more information. 

 

 

Reference 

https://www.cloudera.com/documentation/enterprise/5-9-x/topics/operation_spark_applications.html#con...

avatar
Cloudera Employee

Hi,

 

To view the spark logs of an completed application you can view the logs by running the below command

yarn logs -applicationId application_xxxxxxxxxxxxx_yyyyyy -appOwner <userowner> > application_xxxxxxxxxxxxx_yyyyyy.log

 

Thanks

AKR