Support Questions
Find answers, ask questions, and share your expertise

How to view the log of submitted spark jar program?

How to view the log of submitted spark jar program?

Rising Star

I have submitted a spark java program via "Spark Submit Jar" and it looks running well.

However, when I click the logs link in specified application in Job tab in hue,  it shows "cannot acces: /jobbrowser/jobs/appliacation_****/single_logs."

 

So how can I find logs of running spark application? 

2 REPLIES 2
Highlighted

Re: How to view the log of submitted spark jar program?

Champion

 

You can hit the below url for currently runining jobs for 

 http://spark_driver_host:4040

monitoring purpose 

 

To see the completed jobs you can see in Spark History Web UI 

 

Let me know if you need any more information. 

 

 

Reference 

https://www.cloudera.com/documentation/enterprise/5-9-x/topics/operation_spark_applications.html#con...

Highlighted

Re: How to view the log of submitted spark jar program?

Cloudera Employee

Hi,

 

To view the spark logs of an completed application you can view the logs by running the below command

yarn logs -applicationId application_xxxxxxxxxxxxx_yyyyyy -appOwner <userowner> > application_xxxxxxxxxxxxx_yyyyyy.log

 

Thanks

AKR