How to share data between different oozie actions?

[#1]

Hi,

I have a requirement where I need to share data between multiple mapreduce actions in a single workflow and between different workflows within single coordinator.

I am creating and running my workflows using Hue.

I have read about capture-output element, but this element is not available under mapreduce action. Even if i use Java action to run my mapreduce programs, The capture-element has size constraint of 2KB by default. Could you please let me know how much I can increase the size?

Thanks

[#2 · Accepted solution]

Typically, users use output files on HDFS paths to share data or values between jobs.

The capture-output size is limited cause we store it inside the Oozie DB before we transfer it over to the next action instance. The max size is therefore limited by the size RDBMS supports for CHAR/VARCHAR columns, 64k for MySQL for example.