Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

OOzie Coordinator failing

OOzie Coordinator failing

New Contributor

I wrote a data dependency coordinator job that should name the job output in the date format mentioned in dataset uri. but the job is failing saying that outputDir variable is not resolved. I am posting my coordinator,workflow & job.properties. Let me know where things are going bad. Thanks in advance.

 

error:

2015-11-28 17:40:20,946  INFO ActionStartXCommand:539 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@:start:] Start action [0000033-151128171830337-oozie-bigd-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-11-28 17:40:20,954  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@:start:] [***0000033-151128171830337-oozie-bigd-W@:start:***]Action status=DONE
2015-11-28 17:40:20,954  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@:start:] [***0000033-151128171830337-oozie-bigd-W@:start:***]Action updated in DB!
2015-11-28 17:40:21,336  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@mapreduce-wordcount-example] ELException in ActionStartXCommand 
javax.servlet.jsp.el.ELException: variable [outputDir] cannot be resolved
    at org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java:106)
    at org.apache.commons.el.NamedValue.evaluate(NamedValue.java:124)
    at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
    at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
    at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
    at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:175)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:60)
    at org.apache.oozie.command.XCommand.call(XCommand.java:280)
    at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
    at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
    at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2015-11-28 17:40:21,338  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@mapreduce-wordcount-example] Failing Job due to failed action [mapreduce-wordcount-example]
2015-11-28 17:40:21,340  WARN LiteWorkflowInstance:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@mapreduce-wordcount-example] Workflow Failed. Failing node [mapreduce-wordcount-example]
2015-11-28 17:40:21,431  INFO KillXCommand:539 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000033-151128171830337-oozie-bigd-W
2015-11-28 17:40:21,488  INFO KillXCommand:539 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000033-151128171830337-oozie-bigd-W

coordinator:

  <coordinator-app name="coordinator1" frequency="${frequency}" start="${startTime}" end="${endTime}" timezone="${timezone}" xmlns="uri:oozie:coordinator:0.1">
   <datasets>
     <dataset name="OUT" frequency="${coord:minutes(7)}" initial-instance="2015-11-28T04:00Z" timezone="UTC">
        <uri-template>
         hdfs://localhost:8020/user/bigdata/TimeBasedCoord/Output/${YEAR}-${MONTH}-${DAY}-${HOUR}
        </uri-template>
        <done-flag>_SUCCESS</done-flag>
     </dataset>
   </datasets>

<output-events>
  <data-out name="coordOutData" dataset="OUT">
     <instance>${coord:current(0)}</instance>
   </data-out>
</output-events>

 <action>
  <workflow>
    <app-path>${workflowPath}</app-path>
    <configuration>
      <property>
         <name>outputDir</name>
         <value>${coord:dataOut('coordOutData')}</value>
      </property>
    </configuration>
  </workflow>
 </action>           
</coordinator-app>

Workflow:

<workflow-app xmlns='uri:oozie:workflow:0.1' name='java-main-wf'>
 <start to='mapreduce-wordcount-example' />
   <action name='mapreduce-wordcount-example'>
    <java>
     <job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>default</value> </property> </configuration> <main-class>com.test.mypack.WordCountDriver</main-class> <arg>${inputDir}</arg> <arg>${outputDir}</arg> </java> <ok to="end" /> <error to="fail" />
</action> <kill name="fail"> <message>Java failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name='end' /> </workflow-app>

job.properties:

frequency=10
startTime=2015-11-28T02:00Z
endTime=2015-11-30T10:00Z
timezone=UTC
workflowPath=${nameNode}/user/bigdata/TimeBasedCoord/workflow/
nameNode=hdfs://localhost:8020
jobTracker=localhost:8032
queueName=default
inputDir=${nameNode}/user/${user.name}/TimeBasedCoord/Input
#outputDir=${nameNode}/user/${user.name}/examples/input-data/rawLogs/
oozie.coord.application.path=${nameNode}/user/${user.name}/TimeBasedCoord/coordinator.xml

 

1 REPLY 1
Highlighted

Re: OOzie Coordinator failing

New Contributor

Hi, could you make it work? Thanks