Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here. Want to know more about what has changed? Check out the Community News blog.

Oozie "Multiple "ok to" transitions to the same node are not allowed" in CDH4, but it worked in CDH3

Oozie "Multiple "ok to" transitions to the same node are not allowed" in CDH4, but it worked in CDH3

New Contributor

Hi,

 

I have an oozie workflow that is working fine in CDH3, but it is not even accepted by oozie in CDH4.3. The error message is "Error: E0743 : E0743: Multiple "ok to" transitions to the same node, [smoosh_sdp_fss_reporting_daily], are not allowed".

 

Here is a peuso code of the logic:

START

if (${ wf:conf('join_sdp_fssg') == 'true'} {

ACTION 1

}

ACTION 2

END

 

<decision name="D1">
	<switch>
		<case to="A1">${ wf:conf('variable') == 'true'}</case>
		<default to="A2" />
	</switch>
</decision>
<action name="A1">
	<ok to="A2" />
	<error to="fail" />
</action>
<action name="A2">
	<ok to="end" />
	<error to="fail" />
</action>

 

Here is a sample workflow.xml used to reproduce the problem (it is not the real thing and does not reflect 100% of what is in production):

 

<workflow-app xmlns="uri:oozie:workflow:0.3" name="sdp_fss_reporting_daily">
    <start to="join_sdp_fssg_decision"/>

        <decision name="join_sdp_fssg_decision">
                <switch>
                        <case to="join-sdp-fssg">${ wf:conf('join_sdp_fssg') == 'true'
                                }</case>
                        <default to="smoosh_sdp_fss_reporting_daily" />
                </switch>
        </decision>

    <action name="join-sdp-fssg">
         <java>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
                <prepare>
                        <delete path="${SDP_FSS_JOIN_TMP_DIR}"/>
                        <delete path="${SDP_RAW_FAILURE_OUTPUT_BASE}"/>
                        <delete path="${FSS_RAW_FAILURE_OUTPUT_BASE}"/>
                        <delete path="${SDP_FSS_JOIN_FAILURE_OUTPUT_BASE}"/>
                        <delete path="${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}"/>
                </prepare>
        <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
        </configuration>
                <main-class>net.tmobile.ccr.sdpfssg.SDP_FSSG_Join</main-class>
                <arg>-libJarDir</arg>
                <arg>${LIB_JAR_DIR}</arg>
                <arg>-Dmapred.reduce.tasks=8</arg>
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.TMP_DIR=${SDP_FSS_JOIN_TMP_DIR}</arg>
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.INPUT.SDP_RAW=${SDP_INPUT_BASE}</arg>
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.INPUT.FSSG_RAW=${FSS_INPUT_BASE}</arg>
<!--                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.OUTPUT.SDP=/tmp/alexei/outputs/flume/sdp</arg> -->
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.FAILURE.SDP=${SDP_RAW_FAILURE_OUTPUT_BASE}</arg>
<!--                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.OUTPUT.FSSG=/tmp/alexei/outputs/flume/fssg</arg> -->
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.FAILURE.FSSG=${FSS_RAW_FAILURE_OUTPUT_BASE}</arg>
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.OUTPUT.SDP_FSSG=${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}</arg>
                <arg>-Dnet.tmobile.ccr.sdpfssg.SDP_FSSG_Join.FAILURE.SDP_FSSG=${SDP_FSS_JOIN_FAILURE_OUTPUT_BASE}</arg>
         </java>

        <ok to="smoosh_sdp_fss_reporting_daily"/>

        <error to="fail"/>
    </action>
    <!-- This concatenates and cleans up the part files that are generated by Map Reduce/Pig jobs.-->
    <action name="smoosh_sdp_fss_reporting_daily">
        <sub-workflow>
            <app-path>${nameNode}/workflow/workflows/util/compact</app-path>
            <propagate-configuration/>
            <configuration>
                <property>
                    <name>CLONE_PATH</name>
                    <value>${SDP_FSS_REPORTING_DAILY_TMP_OUTPUT_BASE}/backup</value>
                </property>
                <property>
                    <name>SRC_PATH</name>
                    <value>${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}</value>
                </property>
                <property>
<name>RECURSE</name>
                    <value>false</value>
                </property>
                 <property>
                    <name>smooshType</name>
                    <value>SEQUENCEFILE</value>
                </property>
                <property>
                    <name>TYPE</name>
                    <value>SEQUENCEFILE</value>
                </property>
                <property>
                    <name>DESTINATION_FILE</name>
                    <value>sdp-fss-ccreporting.log</value>
                </property>
                <property>
                    <name>TMP_PATH</name>
                    <value>${SDP_FSS_REPORTING_DAILY_TMP_OUTPUT_BASE}/compact</value>
                </property>

            </configuration>
        </sub-workflow>
        <ok to="copy"/>
        <error to="fail"/>
    </action>
     <action name="copy">
        <pig>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>

            <prepare>
                <delete path="${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}/copy"/>
            </prepare>

            <script>cp.pig</script>

            <param>PARAM_JARFILE=${PARAM_JARFILE}</param>
            <param>SOURCE=${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}/sdp-fss-ccreporting.log</param>
            <param>TARGET=${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}/copy/sdp-fss-ccreporting.log</param>

        </pig>
        <ok to="load-sdp-ccr-2-hive"/>
        <error to="fail"/>
    </action>
 <action name="load-sdp-ccr-2-hive">
        <hive xmlns="uri:oozie:hive-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
<job-xml>/workflow/conf/hive/hive-site.xml</job-xml>
               <configuration>
                    <property>
                        <name>mapred.job.queue.name</name>
                        <value>${queueName}</value>
                    </property>
                </configuration>
            <script>loadsdp-ccreporting.hql</script>
            <param>VARPATH=${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}/copy/sdp-fss-ccreporting.log</param>
            <param>VARYEAR=${JOBYEAR}</param>
            <param>VARMONTH=${JOBMONTH}</param>
            <param>VARDAY=${JOBDAY}</param>

        </hive>
        <ok to="fork-sdp-fss-by-portal-pig"/>
        <error to="end"/>
    </action>

   <fork name="fork-sdp-fss-by-portal-pig">
                <path start="fork-sdp-fss-by-portal-pig-DE"/>
                <path start="fork-sdp-fss-by-portal-pig-UK"/>
      </fork>

    <action name="fork-sdp-fss-by-portal-pig-DE">
        <pig>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
                <property>
                    <name>mapred.compress.map.output</name>
                    <value>true</value>
                </property>
            </configuration>
            <script>SdpFilterByPortal.pig</script>
            <param>PARAM_JARFILE=${PARAM_JARFILE}</param>
            <param>PARAM_SDP=${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}/sdp-fss-ccreporting.log</param>
            <param>PARAM_PORTALID=${SDP_PORTAL_DE}</param>
            <param>PARAM_PARALLEL_MAX=${SDP_FSS_PARAM_PARALLEL_MAX}</param>
            <param>PARAM_OUTPUT_SDP_PORTAL_DIR=${SDP_FSS_REPORTING_DAILY_TMP_OUTPUT_BASE}/${SDP_PORTAL_DE}</param>
        </pig>
        <ok to="end-fork-sdp-fss-by-portal-pig"/>
        <error to="fail"/>
    </action>
    <action name="fork-sdp-fss-by-portal-pig-UK">
        <pig>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
                <property>
                    <name>mapred.compress.map.output</name>
                    <value>true</value>
                </property>
            </configuration>
            <script>SdpFilterByPortal.pig</script>
            <param>PARAM_JARFILE=${PARAM_JARFILE}</param>
            <param>PARAM_SDP=${SDP_FSS_REPORTING_DAILY_OUTPUT_BASE}/sdp-fss-ccreporting.log</param>
            <param>PARAM_PORTALID=${SDP_PORTAL_UK}</param>
            <param>PARAM_PARALLEL_MAX=${SDP_FSS_PARAM_PARALLEL_MAX}</param>
            <param>PARAM_OUTPUT_SDP_PORTAL_DIR=${SDP_FSS_REPORTING_DAILY_TMP_OUTPUT_BASE}/${SDP_PORTAL_UK}</param>
        </pig>
        <ok to="end-fork-sdp-fss-by-portal-pig"/>
        <error to="fail"/>
    </action>

<join name="end-fork-sdp-fss-by-portal-pig" to="end"/>

    <kill name="fail_dependancies">
        <message>SDP CCR Reporting dependancies job failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>


    <kill name="fail">
        <message>SDP CCR Reporting job failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>

    <end name="end"/>

</workflow-app>

 


Thanks,

Alexei

 

 

 

2 REPLIES 2

Re: Oozie "Multiple "ok to" transitions to the same node are not allowed" in CDH

Contributor

Are you sure that that workflow is representative of your original workflow?  E0743 only occurs in the forkjoin checking code (that is, the code that does some extra validation on the nodes between a fork and join).  There is a known issue (recently fixed in OOZIE-1403) where a decision node inside of a fork-join can cause this issue.  Perhaps your original workflow is structured like this?

 

Here's one of the examples from OOZIE-1403:

<fork>
   <goto decision>
   <goto join>
</fork>
<decision>
   <if foo, goto A>
   <else, goto B>
</decision>
<action A>
   <ok to="B">
</action A>
<action B>
   <ok to="join">
</action B>
<join>

 

In the meantime, you can disable the additional forkjoin checking on all workflows by setting oozie.validate.ForkJoin to false in oozie-site.xml or on a per-workflow basis by setting oozie.wf.validate.ForkJoin to false in your job.properties.  

Software Engineer | Cloudera, Inc. | http://cloudera.com
Highlighted

Re: Oozie "Multiple "ok to" transitions to the same node are not allowed" in CDH

New Contributor

Thanks,

 

The big workflow.xml in the original post was the one which caused the problem, if I removed the fork/join, it was accepted by the oozie. But if I left the fork/join and removed the decision action, it passed validation and worked. As you can see the fork-ed actions are ok-ing into a join action.

 

But adding the oozie.validate.ForkJoin=false to job.properties had solved the issue. THANK YOU!!!