About steark

steark · ‎05-03-2016

Well @drussell... I figured as much, but thought I would double check. Thanks for the reply!

steark · ‎05-03-2016

Hi- At current, I am manually cross referencing map IDs from the Oozie web console with YARN logs? This is super tedious and slow... does anyone know of a better way to triage Oozie/YARN logs?

steark · ‎05-03-2016

Thanks Benjamin! The script definitely works locally, and I should have mentioned that. This effort is really just to get the thing fully automated and running without a man in the loop. As for the logs, you are absolutely right... I was able to figure out how to cross reference the Oozie web ui "logs" with the actually Yarn logs via the map ID. Digging into these and talking to some folks, I was able to figure out how to get the curl to run as shell action in hdfs and write out to hdfs. Thanks for the assist! -Kevin

steark · ‎04-29-2016

I'm trying to create a workflow with a command shell action in Oozie and am running into some issues... in the first stage of the workflow, I'm attempting to curl from a local path on an edge node (on a HWX cluster) to an external API and then write results directly into hdfs. However, that action fails with a super not helpful "Launcher ERROR, reason: Main class [org.apache.action.hadoop.ShellMain], exit code [1]" exception... My guess is that either Oozie can't see both the local path (to call the curl) and hdfs (to write out results) or that curl (via the -o argument) won't support writing to hdfs... any ideas? Thanks- Kevin

Online	Offline
Last Visited	‎06-22-2016 09:20 PM

Member Since	‎12-15-2015 04:43 PM
Last Visited	‎06-22-2016 09:20 PM
Posts	6

Cloudera Community

Re: There has to be a better way to triage Oozie l...

There has to be a better way to triage Oozie logs?...

Re: Can Oozie curl from an external API and write ...

Can Oozie curl from an external API and write to H...