- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
How to read the parameter value in mapper code passed from oozie workflow?
- Labels:
-
Apache Oozie
Created on 07-27-2015 02:58 AM - edited 09-16-2022 02:35 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I am running my oozie workflow through command prompt.
I assigning value to a parameter "var" in my my workflow.
<property>
<name>var</name>
<value>2</value>
</property>
Now how can i access the variable value in my java Mapper Program.
Will the below metioned work for me:
protected void setup(Context context) throws IOException,InterruptedException {
Configuration conf = context.getConfiguration();
var_val = conf.get("var");
}
Created 07-27-2015 03:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
configuration property. If you are using a java action type instead, you
will also need to load the configuration in the driver explicitly:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action
"""
A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added
to the section
of the java action. The Hadoop configuration file is made available as a
local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie-action.conf.xml . For example:
// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));
If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
"""
Created 07-27-2015 03:03 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
configuration property. If you are using a java action type instead, you
will also need to load the configuration in the driver explicitly:
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action
"""
A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added
to the section
of the java action. The Hadoop configuration file is made available as a
local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie-action.conf.xml . For example:
// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));
If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
"""
Created 07-28-2015 06:43 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Harsh,
The information provided for Java Action is clear to me. But I have doubt regarding MapReduce action.
I am using only Mapreduce action for my workflow and I am placing only my Mapper class & Reducer Class in the .jar file. I want to pass all the properties and parameters through Oozie workflow.
Now can I get the value of the "var" variable, i.e. "2", in my Mapper using the below code in my mapper class.
protected void setup(Context context) throws IOException,InterruptedException {
Configuration conf = context.getConfiguration();
var_val = conf.get("var");
}
Created 07-28-2015 07:06 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You'll need to pass the XML property via the workflow.xml under the
action's configuration section.