Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Spark 1.4.1 logs through history server not present

Highlighted

Spark 1.4.1 logs through history server not present

Explorer

Running spark-submit job with Spark 1.4.1 on HDP 2.3.2. Jobs show no logs when I click on the Executors tab for my spark job. Jobs and Stages tabs are also empty. Where can I find the stderr/out logs for my job?

Also, job still says "Incomplete" when it has finished.

This same app works fine in Spark 1.6.1. I can see everything and can refresh the page as it runs with updates.

1 REPLY 1
Highlighted

Re: Spark 1.4.1 logs through history server not present

Mentor

Consider upgrading your cluster but in the meantime, please review the Spark 1.4.1 configurations from this doc http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_spark-guide/content/ch_tuning-spark.html#...

It does not hurt to review ATS doc also for any missing properties http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/ch_timeline_ser...

Don't have an account?
Coming from Hortonworks? Activate your account here