Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Run oozie java action on remote cluster

Run oozie java action on remote cluster

New Contributor

I'm trying to use oozie and copyTable to tranfser Hbase data between two kerborized clusters. My workflow works fine in both directions but only when it is started on source side. What I need is to run the entire workflow on target but launch the copyTable action on souce (remote). When I assign the RM and NN from remote cluster to my copyTable action the workflow starts a launcher job on remote cluster which fails due to invalid deligation token. The log tells me that the launcher job tries to (re)connect to RM on target (where the workflow was started).

 

Is it possible to run a workflow in that way? What could be reason why it fails? 

2 REPLIES 2

Re: Run oozie java action on remote cluster

Master Guru
While Oozie can be configured to talk to multiple HDFS and YARN clusters, the CM configuration does not cater this out of the box. The Oozie in CM is bound to one set of HDFS and YARN clusters, as linked on the Configuration page for it.

Extending it, while straight forward, involves manually configuring the oozie.service.HadoopAccessorService.hadoop.configurations property in oozie-site.xml to map each host authority (of RM and NameNode hosts/nameservices of each cluster) to its own configuration directory locations.

Have you already done the above, prior to attempting this? How are you specifically configuring your Oozie actions to submit to remote RMs? Can you share your workflow.xml and submit-time properties?

Re: Run oozie java action on remote cluster

New Contributor

Meanwhile I found the solution. In addtion to the whitelisting in oozie.service, you have to create hbase credentials for source and target and to assign both to java action. Furthermore you must load hbase-xml from source cluster and configure RMS and JHS. Then it works fine!

Don't have an account?
Coming from Hortonworks? Activate your account here