Workaround for log aggregation bug

[#1]

We have log aggregation enabled in the Yarn configuration for our cluster (yarn.log-aggregation-enable).

But it doesn't seem to work.

When I try to drill into the history of a job in the resource manager GUI, the link for "logs" always takes me to a page that says: "aggregation is not enabled".

I've opened a ticket asking for help on this, and they told us we need to upgrade, so we did, but it didn't help.

I opened another ticket and am currently waiting for a response.

In the meantime, has anyone seen this?

Is there is a known hack to fix it?

Any advice about where to look for the solution?

We're currently on 2.2.8

[#2]

@Zack Riesland

did you follow this doc? https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_yarn_resource_mgt/content/ch_log_aggrega...

this should also give you some info http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/

[#3]

Yes. It basically just says to set two config properties, which I did.

[#4]

I remember having this issue until I configured TimeLine Server. @Zack Riesland

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_yarn_resource_mgt/content/ch_timeline_ser...

[#5]

Latest doc @Zack Riesland

http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_yarn_resource_mgt/content/ref-ca0dc265-90...

[#6]

Thanks Artem,

Yes, this all seems to be setup correctly. I believe we had to tackle this a while back to get the Tez View working.

[#7]

And you migrated to 1.5 of TS? @Zack Riesland

[#8]

I'm not explicitly specifying yarn.timeline-service.version, so I guess I'm running whatever the 2.2.8 default is.

Do you think I'll have better luck if I upgrade?

[#9]

I would either wait for support or try in dev cluster. @Zack Riesland

[#10]

@Zack Riesland

Please see this thread https://community.hortonworks.com/questions/4719/yarn-logs-seems-to-be-truncated.html

I am afraid that you may not find something really useful. Please followup with the support suggestion.