Created 03-15-2017 08:26 PM
An oozie sub-workflow is failing with message as "LauncherMapper died, check Hadoop LOG for job"
On subsequent attempt, the workflow completes successfully.
Various oozie jobs are failing randomly at any sub-workflow and work fine on subsequent attempt.
I tried checking hadoop logs but when the failure happens there are no mapred or yarn logs for that job/application.
Any hints are highly appreciated.
Created 03-21-2017 06:56 PM
Restart yarn & oozie services and then try to run your oozie workflow again. I had a similar situation and I tried many things, in the end the restart solved the problem for me.
Created 03-15-2017 10:33 PM
Please check whether failures only occur on the exact same node? Also can you drill down into the yarn job logs and see what error you get?
Created 03-16-2017 03:53 AM
@Artem Ervits - different workflows failed on different nodes, however, their re-attempt were on the same node as their first attempt.
also the yarn logs do not exist for the failed/killed ones, i mean, i tried > yarn logs -applicationId <failedAppId> but it said the logs do not exist.
In order to verify the command i tried to get logs for a successful one and was able to get logs.
Thank you.
Created 03-21-2017 06:56 PM
Restart yarn & oozie services and then try to run your oozie workflow again. I had a similar situation and I tried many things, in the end the restart solved the problem for me.
Created 03-21-2017 07:06 PM
It is weird but a restart indeed solved the issue. Wondering what the root cause could be. If you have any luck arriving at the root cause, please share here in future. 🙂