Reply
New Contributor
Posts: 3
Registered: ‎11-28-2015

OOzie Coordinator failing

I wrote a data dependency coordinator job that should name the job output in the date format mentioned in dataset uri. but the job is failing saying that outputDir variable is not resolved. I am posting my coordinator,workflow & job.properties. Let me know where things are going bad. Thanks in advance.

 

error:

2015-11-28 17:40:20,946  INFO ActionStartXCommand:539 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@:start:] Start action [0000033-151128171830337-oozie-bigd-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10]
2015-11-28 17:40:20,954  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@:start:] [***0000033-151128171830337-oozie-bigd-W@:start:***]Action status=DONE
2015-11-28 17:40:20,954  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@:start:] [***0000033-151128171830337-oozie-bigd-W@:start:***]Action updated in DB!
2015-11-28 17:40:21,336  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@mapreduce-wordcount-example] ELException in ActionStartXCommand 
javax.servlet.jsp.el.ELException: variable [outputDir] cannot be resolved
    at org.apache.oozie.util.ELEvaluator$Context.resolveVariable(ELEvaluator.java:106)
    at org.apache.commons.el.NamedValue.evaluate(NamedValue.java:124)
    at org.apache.commons.el.ExpressionString.evaluate(ExpressionString.java:114)
    at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:274)
    at org.apache.commons.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:190)
    at org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:203)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:175)
    at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:60)
    at org.apache.oozie.command.XCommand.call(XCommand.java:280)
    at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:326)
    at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:255)
    at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
2015-11-28 17:40:21,338  WARN ActionStartXCommand:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@mapreduce-wordcount-example] Failing Job due to failed action [mapreduce-wordcount-example]
2015-11-28 17:40:21,340  WARN LiteWorkflowInstance:542 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[0000033-151128171830337-oozie-bigd-W@mapreduce-wordcount-example] Workflow Failed. Failing node [mapreduce-wordcount-example]
2015-11-28 17:40:21,431  INFO KillXCommand:539 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000033-151128171830337-oozie-bigd-W
2015-11-28 17:40:21,488  INFO KillXCommand:539 - USER[bigdata] GROUP[-] TOKEN[] APP[java-main-wf] JOB[0000033-151128171830337-oozie-bigd-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000033-151128171830337-oozie-bigd-W

coordinator:

  <coordinator-app name="coordinator1" frequency="${frequency}" start="${startTime}" end="${endTime}" timezone="${timezone}" xmlns="uri:oozie:coordinator:0.1">
   <datasets>
     <dataset name="OUT" frequency="${coord:minutes(7)}" initial-instance="2015-11-28T04:00Z" timezone="UTC">
        <uri-template>
         hdfs://localhost:8020/user/bigdata/TimeBasedCoord/Output/${YEAR}-${MONTH}-${DAY}-${HOUR}
        </uri-template>
        <done-flag>_SUCCESS</done-flag>
     </dataset>
   </datasets>

<output-events>
  <data-out name="coordOutData" dataset="OUT">
     <instance>${coord:current(0)}</instance>
   </data-out>
</output-events>

 <action>
  <workflow>
    <app-path>${workflowPath}</app-path>
    <configuration>
      <property>
         <name>outputDir</name>
         <value>${coord:dataOut('coordOutData')}</value>
      </property>
    </configuration>
  </workflow>
 </action>           
</coordinator-app>

Workflow:

<workflow-app xmlns='uri:oozie:workflow:0.1' name='java-main-wf'>
 <start to='mapreduce-wordcount-example' />
   <action name='mapreduce-wordcount-example'>
    <java>
     <job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>default</value> </property> </configuration> <main-class>com.test.mypack.WordCountDriver</main-class> <arg>${inputDir}</arg> <arg>${outputDir}</arg> </java> <ok to="end" /> <error to="fail" />
</action> <kill name="fail"> <message>Java failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name='end' /> </workflow-app>

job.properties:

frequency=10
startTime=2015-11-28T02:00Z
endTime=2015-11-30T10:00Z
timezone=UTC
workflowPath=${nameNode}/user/bigdata/TimeBasedCoord/workflow/
nameNode=hdfs://localhost:8020
jobTracker=localhost:8032
queueName=default
inputDir=${nameNode}/user/${user.name}/TimeBasedCoord/Input
#outputDir=${nameNode}/user/${user.name}/examples/input-data/rawLogs/
oozie.coord.application.path=${nameNode}/user/${user.name}/TimeBasedCoord/coordinator.xml

 

New Contributor
Posts: 2
Registered: ‎02-23-2017

Re: OOzie Coordinator failing

Hi, could you make it work? Thanks

Announcements