Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to read the parameter value in mapper code passed from oozie workflow?

SOLVED Go to solution

How to read the parameter value in mapper code passed from oozie workflow?

Explorer

Hi,

 

I am running my oozie workflow through command prompt.

 

I assigning value to a parameter "var" in my my workflow.

 

       <property>
              <name>var</name>
              <value>2</value>
       </property>

 

Now how can i access the variable value in my java Mapper Program.

 

Will the below metioned work for me:

 

protected void setup(Context context) throws IOException,InterruptedException {

Configuration conf = context.getConfiguration();

var_val = conf.get("var");

}

 

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to read the parameter value in mapper code passed from oozie workflow

Master Guru
Yes, that will work if you use a map-reduce action type to define that
configuration property. If you are using a java action type instead, you
will also need to load the configuration in the driver explicitly:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

"""

A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added
to the section
of the java action. The Hadoop configuration file is made available as a
local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie-action.conf.xml . For example:

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));

If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
"""
3 REPLIES 3
Highlighted

Re: How to read the parameter value in mapper code passed from oozie workflow

Master Guru
Yes, that will work if you use a map-reduce action type to define that
configuration property. If you are using a java action type instead, you
will also need to load the configuration in the driver explicitly:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action

"""

A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added
to the section
of the java action. The Hadoop configuration file is made available as a
local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie-action.conf.xml . For example:

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));

If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
"""

Re: How to read the parameter value in mapper code passed from oozie workflow

Explorer

Hi Harsh,

 

The information provided for Java Action is clear to me. But I have doubt regarding MapReduce action.

 

I am using only Mapreduce action for my workflow and I am placing only my Mapper class & Reducer Class in the .jar file. I want to pass all the properties and parameters through Oozie workflow.

 

Now can I get the value of the "var" variable, i.e. "2", in my Mapper using the below code in my mapper class.

 

protected void setup(Context context) throws IOException,InterruptedException {

Configuration conf = context.getConfiguration();

var_val = conf.get("var");

}

Re: How to read the parameter value in mapper code passed from oozie workflow

Master Guru
Yes, do you not see it working?

You'll need to pass the XML property via the workflow.xml under the
action's configuration section.