Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to view the log of submitted spark jar program?

avatar
Rising Star

I have submitted a spark java program via "Spark Submit Jar" and it looks running well.

However, when I click the logs link in specified application in Job tab in hue,  it shows "cannot acces: /jobbrowser/jobs/appliacation_****/single_logs."

 

So how can I find logs of running spark application? 

2 REPLIES 2

avatar
Champion

 

You can hit the below url for currently runining jobs for 

 http://spark_driver_host:4040

monitoring purpose 

 

To see the completed jobs you can see in Spark History Web UI 

 

Let me know if you need any more information. 

 

 

Reference 

https://www.cloudera.com/documentation/enterprise/5-9-x/topics/operation_spark_applications.html#con...

avatar
Cloudera Employee

Hi,

 

To view the spark logs of an completed application you can view the logs by running the below command

yarn logs -applicationId application_xxxxxxxxxxxxx_yyyyyy -appOwner <userowner> > application_xxxxxxxxxxxxx_yyyyyy.log

 

Thanks

AKR