Support Questions
Find answers, ask questions, and share your expertise

Log aggregation timeout for spark jobs in yarn

Expert Contributor

Looks like logs are not completely aggregated to the application and there may be truncation of logs too .we can see from RM that for the application there is timeout for some of the nodemanagers . Job runs for 55 minutes and we have set yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds to 3600. Any Pointers.

1 REPLY 1

Explorer

i am facing same issue , did you get solution for this

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.