Reply
Explorer
Posts: 7
Registered: ‎03-01-2019

How to add jars in classpath in mapper code passed from oozie workflow java action?

[ Edited ]

I have the following problem. I have added hive jars in java action of oozie using shared lib. oozie.action.sharelib.for.java = hive.

After this i am able to get the jars in class path in the java class that runs using oozie. But i am not able to get these jars in mapper created by above java class.

How should i add these jars in class path of these mappers.

Posts: 1,893
Kudos: 432
Solutions: 302
Registered: ‎07-31-2013

Re: How to add jars in classpath in mapper code passed from oozie workflow java action?

For plain Java actions, Oozie cannot automatically/transparently setup a
usable environment, and therefore relies on the developers to manually
configure it. It does, however, help the developers with a pre-prepared
configuration file that they can simply absorb into their class's
invocation.

Here's the relevant part from the documentation you'll need to follow when
building your driver for use in Oozie Java actions:

"""
A java action can create a Hadoop configuration for interacting with a
cluster (e.g. launching a map-reduce job). Oozie prepares a Hadoop
configuration file which includes the environments site configuration files
(e.g. hdfs-site.xml, mapred-site.xml, etc) plus the properties added to the
section of the java action. The Hadoop configuration file is made available
as a local file to the Java application in its running directory. It can be
added to the java actions Hadoop configuration by referencing the system
property: oozie.action.conf.xml . For example:

// loading action conf prepared by Oozie
Configuration actionConf = new Configuration(false);
actionConf.addResource(new Path("file:///",
System.getProperty("oozie.action.conf.xml")));

If oozie.action.conf.xml is not added then the job will pick up the
mapred-default properties and this may result in unexpected behaviour. For
repeated configuration properties later values override earlier ones.
""" -
http://archive.cloudera.com/cdh5/cdh/5/oozie/WorkflowFunctionalSpec.html#a3.2.7_Java_Action
Announcements