question Re: Passing Paramter to Oozie workflow in Archives of Support Questions (Read Only)

Passing Paramter to Oozie workflow

Anishsk10 — Fri, 16 Sep 2022 09:33:48 GMT

Hi,

I am new Oozie and some doubts regarding the creating workflows.

I have java mapreduce progrm which expects 3 parameters to be passed: input directory, output directory and the field position of the record in the file that I need to pass to the program as parameter.

hadoop jar /home/cloudera/ooziesample.jar /user/cloudera/wordcount/input/ /user/cloudera/wordcount/output 2

Now I want to create a workflow for running this jar as mapreduce actions. I want to pass these 3 parameters via oozie workflow : input directory, output directory and the field position of the record. Could you please help me know how to do the same

Re: Passing Paramter to Oozie workflow

Harsh J — Wed, 15 Jul 2015 05:08:28 GMT

You can use the new Java driver feature of the Oozie MR action to run your jar in an easier way. Follow this: http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.2.2_Configuring_the_MapReduce_action_with_Java_code

Re: Passing Paramter to Oozie workflow

Anishsk10 — Thu, 16 Jul 2015 13:17:54 GMT

Hi,

Thanks for the reply.

I am trying to use Hue for creating and running the Oozie workflows.

Whatever Properties that I am trying to pass (eg: mapreduce.map.class and mapreduce.reduce.class) is not getting written to job.properties file. Hence could see those in workflow.xml as well.

I have few Qs:

Where should define parameter and its value in Hue UI for a workflow?

Do I need to upload .jar in lib directory of the workflow and what is the need of that?

My Use-case:

I want to run the java mapreduce program using mapreduce action.

I have a jar with multiple Mapper and Reducer Class defined and I want to use Oozie as driver for my Mapper and Reducer programs. And at the same time passing argument values through Oozie to my Mappers and Reducers.

Could you please let me know how to achieve this using Hue and Oozie?

Thanks

Re: Passing Paramter to Oozie workflow

Harsh J — Thu, 16 Jul 2015 13:36:18 GMT

> is not getting written to job.properties file

The job.properties file is just for WF variable resolution and some Oozie submission control properties. They aren't used for actual job configuration, however.

> Where should define parameter and its value in Hue UI for a workflow?

Within your Action's configuration view, look for "Properties" to see a Key=Value like field. Enter your configuration properties here.

If you are using new-API values, you may want to read the requirements also at https://cwiki.apache.org/confluence/display/OOZIE/Map+Reduce+Cookbook (search page for "new-api").

> Do I need to upload .jar in lib directory of the workflow and what is the need of that?

You will need to upload the application and any non-hadoop dependencies, because Oozie runs a remote launcher job to run your classes, and the jars need to exist within the workflow lib directory for Oozie to ship them properly via distributed-cache.

Does this help?

Re: Passing Paramter to Oozie workflow

Anishsk10 — Thu, 16 Jul 2015 14:10:58 GMT

Hi,

Your reply does help. Thanks.

But I still have doubts regarding passing parameters via Hue UI.

1.If I pass the parameter value using the Properties field, then will it be updated in the workflow under <agr> tag or <property> tag?

2. What is the use of Parameter field under Workflow Settings.

Also could you please give some insight regarding my use-case: I want to run the java mapreduce program using mapreduce action.

How can I achieve this using Hue UI to create and workflow?