question Re: Oozie - Is there a way to capture output of Sqoop eval node in other oozie nodes ? in Archives of Support Questions (Read Only)

Oozie - Is there a way to capture output of Sqoop eval node in other oozie nodes ?

shishir_saxena4 — Tue, 01 Mar 2016 12:04:21 GMT

I have an Oozie job that is performing sqoop import action. I want to add another sqoop eval node to this job to run sqoop eval query on source database to get control totals and insert control totals to another Hive table. Is there a way to capture output of sqoop eval node in Oozie ?

Re: Oozie - Is there a way to capture output of Sqoop eval node in other oozie nodes ?

bleonhardi — Tue, 01 Mar 2016 17:06:56 GMT

I don't think so. there is no capture output attribute in the sqoop action XML. However you could run a sqoop command in a shell or ssh action ( the latter I did the first might need the sqoop libraries added which might be more setup). You could then capture the output of the shell or ssh action. Essentially run sqoop in a shellscript and grep the output you want and put it into key=value pairs. Those you can then use in the following action eval for example.

Re: Oozie - Is there a way to capture output of Sqoop eval node in other oozie nodes ?

shishir_saxena4 — Tue, 01 Mar 2016 19:43:05 GMT

Thank You @Benjamin Leonhardi. That was my thought process also. I am planning to skip Oozie and just run my sqoop scripts, capture output and grep for keys. Thanks for validating it.

Re: Oozie - Is there a way to capture output of Sqoop eval node in other oozie nodes ?

MilesYao — Wed, 02 Mar 2016 03:19:49 GMT

Same issue with Hive action. It is frustrating and inefficient that such a common use case cannot be accommodated.