Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oozie coordinator timezone

Solved Go to solution
Highlighted

Oozie coordinator timezone

Contributor

Hi all,

I've created an Oozie coordinator with synchronous dataset. The time in the cluster is set to CEST (GMT+2). I'm using flume to collect data and create a directory in HDFS in this format:

/flume/%Y/%m/%d/%H

coordinator.properties:

nameNode=hdfs://vm1.local:8020
jobTracker=vm1.local:8050
queueName=default
exampleDir=${nameNode}/user/root/oozie-wait

oozie.use.system.libpath = true

start=2016-08-03T08:01Z
end=2016-08-03T12:06Z
workflowAppUri=${exampleDir}/app

oozie.coord.application.path=${exampleDir}/app

coordiantor.xml:

<coordinator-app name="every-hour-waitForData" frequency="${coord:hours(1)}" start="${start}" end="${end}" timezone="UTC"
                 xmlns="uri:oozie:coordinator:0.1">
<datasets>    
<dataset name="ratings" frequency="${coord:hours(1)}" initial-instance="${start}" timezone="Europe/Paris">
    <uri-template>hdfs://vm1.local:8020/user/root/flume/${YEAR}/${MONTH}/${DAY}/${HOUR}</uri-template>
  </dataset>
</datasets>

<input-events>
      <data-in name="coordInput1" dataset="ratings">
        <instance>${coord:current(0)}</instance>
      </data-in>
   </input-events>

        <action>
        <workflow>
            <app-path>${workflowAppUri}</app-path>
            <configuration>
        <property>
              <name>wfInput</name>
              <value>${coord:dataIn('coordInput1')}</value>
            </property>
                <property>
                    <name>jobTracker</name>
                    <value>${jobTracker}</value>
                </property>
                <property>
                    <name>nameNode</name>
                    <value>${nameNode}</value>
                </property>
                <property>
                    <name>queueName</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
        </workflow>
    </action>
</coordinator-app>

When running this example flume creates the directory

/user/root/flume/2016/08/03/10/

But the coordinator is waiting for /user/root/flume/2016/08/03/08

Does any one knows how to make Flume creates the directory in UTC or the coordinator reads the correct directory .

Thanks.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Oozie coordinator timezone

Contributor

Okay, I found a workaround, I added:

-Duser.timezone=GMT

which changes the the JVM timezon. The final Flume-ng command will be as following:

flume-ng agent --conf-file spool1.properties --name agent1 --conf $FLUME_HOME/conf -Duser.timezone=GMT

The needed directory for the oozie coordiantor is now being created.

View solution in original post

1 REPLY 1

Re: Oozie coordinator timezone

Contributor

Okay, I found a workaround, I added:

-Duser.timezone=GMT

which changes the the JVM timezon. The final Flume-ng command will be as following:

flume-ng agent --conf-file spool1.properties --name agent1 --conf $FLUME_HOME/conf -Duser.timezone=GMT

The needed directory for the oozie coordiantor is now being created.

View solution in original post

Don't have an account?
Coming from Hortonworks? Activate your account here