Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Oozie coordinator timezone

avatar
Rising Star

Hi all,

I've created an Oozie coordinator with synchronous dataset. The time in the cluster is set to CEST (GMT+2). I'm using flume to collect data and create a directory in HDFS in this format:

/flume/%Y/%m/%d/%H

coordinator.properties:

nameNode=hdfs://vm1.local:8020
jobTracker=vm1.local:8050
queueName=default
exampleDir=${nameNode}/user/root/oozie-wait

oozie.use.system.libpath = true

start=2016-08-03T08:01Z
end=2016-08-03T12:06Z
workflowAppUri=${exampleDir}/app

oozie.coord.application.path=${exampleDir}/app

coordiantor.xml:

<coordinator-app name="every-hour-waitForData" frequency="${coord:hours(1)}" start="${start}" end="${end}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.1">
<datasets>    
<dataset name="ratings" frequency="${coord:hours(1)}" initial-instance="${start}" timezone="Europe/Paris">
    <uri-template>hdfs://vm1.local:8020/user/root/flume/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
  </dataset>
</datasets>

<input-events>
      <data-in name="coordInput1" dataset="ratings">
        <instance>${coord:current(0)}</instance>
      </data-in>
   </input-events>

        <action>
        <workflow>
            <app-path>${workflowAppUri}</app-path>
            <configuration>
        <property>
              <name>wfInput</name>
              <value>${coord:dataIn('coordInput1')}</value>
            </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>queueName</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

When running this example flume creates the directory

/user/root/flume/2016/08/03/10/

But the coordinator is waiting for /user/root/flume/2016/08/03/08

Does any one knows how to make Flume creates the directory in UTC or the coordinator reads the correct directory .

Thanks.

1 ACCEPTED SOLUTION

avatar
Rising Star

Okay, I found a workaround, I added:

-Duser.timezone=GMT

which changes the the JVM timezon. The final Flume-ng command will be as following:

flume-ng agent --conf-file spool1.properties --name agent1 --conf $FLUME_HOME/conf -Duser.timezone=GMT

The needed directory for the oozie coordiantor is now being created.

View solution in original post

1 REPLY 1

avatar
Rising Star

Okay, I found a workaround, I added:

-Duser.timezone=GMT

which changes the the JVM timezon. The final Flume-ng command will be as following:

flume-ng agent --conf-file spool1.properties --name agent1 --conf $FLUME_HOME/conf -Duser.timezone=GMT

The needed directory for the oozie coordiantor is now being created.