Yes, I'm using the Oozie streaming action. And streaming jar is the one which is bundled with CDH. What is strange is that we use the same workflows and coordinators in our production cluster which is on CDH 5.1. Here is example workflow action we use: <action name="raw-pass" retry-max="3" retry-interval="1">
<map-reduce>
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<prepare>
<delete path="${rawPassOutput}"/>
</prepare>
<streaming>
<mapper>mapreducers/bin/mapred run MyJobMapper</mapper>
<reducer>mapreducers/bin/mapred run MyJobReducer</reducer>
</streaming>
<configuration>
<property>
<!--
This will add avro.jar and avro-mapred.jar dependencies to the job
(@see mapred.input.format.class property below)
-->
<name>oozie.action.sharelib.for.map-reduce</name>
<value>mapreduce-streaming,hcatalog,sqoop</value>
</property>
<property>
<name>mapred.reduce.tasks</name>
<value>1</value>
</property>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
<property>
<name>mapred.input.dir</name>
<value>${rawPassInput}</value>
</property>
<property>
<name>mapred.output.dir</name>
<value>${rawPassOutput}</value>
</property>
<property>
<!--
This input format will automagically decode avro files so
that our mappers will get plain json as input.
-->
<name>mapred.input.format.class</name>
<value>org.apache.avro.mapred.AvroAsTextInputFormat</value>
</property>
</configuration>
<archive>${mapreducersArchive}#mapreducers</archive>
</map-reduce>
<ok to="aggregate-pass"/>
<error to="failure-email-notification"/>
</action> I can not share actual mapper and reducer scripts though. Also - is there a relatively easy way to downgrade CDH installation from CDH 5.2 to 5.1? Or at least to install 5.1 from scratch? I'm not sure there was an option in Cloudera Manager to install 5.1, only 5.2... Thanks!
... View more
