Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to read the parameter value in mapper code passed from oozie workflow?

avatar
Explorer

Hi,

 

I am running my oozie workflow through command prompt.

 

I assigning value to a parameter "var" in my my workflow.

 

       <property>
              <name>var</name>
              <value>2</value>
       </property>

 

Now how can i access the variable value in my java Mapper Program.

 

Will the below metioned work for me:

 

protected void setup(Context context) throws IOException,InterruptedException {

Configuration conf = context.getConfiguration();

var_val = conf.get("var");

}

 

1 ACCEPTED SOLUTION

avatar
Mentor
Yes, that will work if you use a map-reduce action type to define that
configuration property. If you are using a java action type instead, you
will also need to load the configuration in the driver explicitly:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

"""

A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added
to the section
of the java action. The Hadoop configuration file is made available as a
local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie-action.conf.xml . For example:

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));

If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
"""

View solution in original post

3 REPLIES 3

avatar
Mentor
Yes, that will work if you use a map-reduce action type to define that
configuration property. If you are using a java action type instead, you
will also need to load the configuration in the driver explicitly:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

"""

A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added
to the section
of the java action. The Hadoop configuration file is made available as a
local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie-action.conf.xml . For example:

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));

If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
"""

avatar
Explorer

Hi Harsh,

 

The information provided for Java Action is clear to me. But I have doubt regarding MapReduce action.

 

I am using only Mapreduce action for my workflow and I am placing only my Mapper class & Reducer Class in the .jar file. I want to pass all the properties and parameters through Oozie workflow.

 

Now can I get the value of the "var" variable, i.e. "2", in my Mapper using the below code in my mapper class.

 

protected void setup(Context context) throws IOException,InterruptedException {

Configuration conf = context.getConfiguration();

var_val = conf.get("var");

}

avatar
Mentor
Yes, do you not see it working?

You'll need to pass the XML property via the workflow.xml under the
action's configuration section.