Oozie coordinator timezone

zaher_mahdhi — Wed, 03 Aug 2016 15:43:56 GMT

Hi all,

I've created an Oozie coordinator with synchronous dataset. The time in the cluster is set to CEST (GMT+2). I'm using flume to collect data and create a directory in HDFS in this format:

/flume/%Y/%m/%d/%H

coordinator.properties:

nameNode=hdfs://vm1.local:8020
jobTracker=vm1.local:8050
queueName=default
exampleDir=${nameNode}/user/root/oozie-wait

oozie.use.system.libpath = true

start=2016-08-03T08:01Z
end=2016-08-03T12:06Z
workflowAppUri=${exampleDir}/app

oozie.coord.application.path=${exampleDir}/app

coordiantor.xml:

<coordinator-app name="every-hour-waitForData" frequency="${coord:hours(1)}" start="${start}" end="${end}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.1">
<datasets>    
<dataset name="ratings" frequency="${coord:hours(1)}" initial-instance="${start}" timezone="Europe/Paris">
    <uri-template>hdfs://vm1.local:8020/user/root/flume/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
  </dataset>
</datasets>

<input-events>
      <data-in name="coordInput1" dataset="ratings">
        <instance>${coord:current(0)}</instance>
      </data-in>
   </input-events>

        <action>
        <workflow>
            <app-path>${workflowAppUri}</app-path>
            <configuration>
        <property>
              <name>wfInput</name>
              <value>${coord:dataIn('coordInput1')}</value>
            </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>queueName</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

When running this example flume creates the directory

/user/root/flume/2016/08/03/10/

But the coordinator is waiting for /user/root/flume/2016/08/03/08

Does any one knows how to make Flume creates the directory in UTC or the coordinator reads the correct directory .

Thanks.

Re: Oozie coordinator timezone

zaher_mahdhi — Wed, 03 Aug 2016 21:49:38 GMT

Okay, I found a workaround, I added:

-Duser.timezone=GMT

which changes the the JVM timezon. The final Flume-ng command will be as following:

flume-ng agent --conf-file spool1.properties --name agent1 --conf $FLUME_HOME/conf -Duser.timezone=GMT

The needed directory for the oozie coordiantor is now being created.

question Re: Oozie coordinator timezone in Archives of Support Questions (Read Only)

Oozie coordinator timezone

Re: Oozie coordinator timezone