question Re: What are the best practices for implementing, testing and deploying oozie custom actions? in Support Questions

What are the best practices for implementing, testing and deploying oozie custom actions?

BillM. — Fri, 16 Sep 2022 09:29:15 GMT

We have a preliminary implementation of an oozie custom action (asyncronous if it matters) and would like to do some unit and functional testing prior without disturbing our production cluster and its workflows.

What approaches are recommended for testing and validation of oozie custom actions?
What debugging approaches are recommended?
Can we test in a vm sandbox (or in native mode on our Mac OSX dev machines) instead of on a cluster?
Are there any deployment approaches/issues that we should be aware of?

I tried posting this earlier, but didn't see it post, so please forgive me if it shows up more than once.

Re: What are the best practices for implementing, testing and deploying oozie custom actions?

Robert K — Thu, 21 May 2015 16:46:16 GMT

Hi,

To test a custom action, I'd recommend simply running it in an Oozie server and seeing if you run into any problems. You can also attach a debugger to Oozie to help there. To save on turn-around time, you can also write a unit test that starts up LocalOozie and tries to submit your action. To attach a debugger to Oozie, you should add the following to the "Oozie Service Environment Advanced Configuration Snippet (Safety Valve)" in Cloudera Manager (if not using CM, then you'd export this in oozie-env.sh:
```
CATALINA_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005"
```
Besides using the debugger I mentioned above, looking at the launcher output (assuming you added print statements) can also be helpful; you can attach a debugger to the launcher job, but that's a bit harder because you need to attach it to the MR container instead of the Oozie server.
Oozie is pretty easy to run anywhere. You can run it in a VM, on a cluster, or even on your Mac. When developing for Oozie, I typically run it on my Mac against a pseudo-distributed cluster, also on my Mac. You can use oozie.service.HadoopAccessorService.hadoop.configurations to point Oozie at any Hadoop cluster, or even multiple ones.
Nothing special. Just make sure that your custom action's jar is in the Oozie servers classpath and that if your custom action has a sharelib, it's deployed in HDFS and Oozie sees it.

Re: What are the best practices for implementing, testing and deploying oozie custom actions?

Dat Pham — Wed, 03 Jun 2015 18:27:05 GMT

1. Oozie "validate" is not supported with custom action so this would need to be commented out. 2. need to add the custom action class [a] and schema [b] via CM: [a] Oozie -> Configuration -> Oozie ActionService Executor Extension Classes (add "org.apache.oozie.action.hadoop.MyShellActionExecutor") [b] Oozie -> Configuration -> Oozie SchemaService Workflow Extension Schemas (add "my-shell-action-0.1.xsd") 3. put the custom jar under /var/lib/oozie 4. restart Oozie 5. once Oozie has been restarted, go to the Oozie to verify that settings are applied: CM -> Oozie -> Oozie Web UI -> System Info (search for "MyShellActionExecutor" and " my-shell-action-0.1.xsd)

Re: What are the best practices for implementing, testing and deploying oozie custom actions?

Robert K — Wed, 03 Jun 2015 23:54:28 GMT

By the way, OOZIE-2159 will fix the 'oozie validate' command not supporting custom actions by moving the check from the Oozie client to the Oozie server, where the custom action is configured.