Skip to main content
KeithLeoIdeam
KeithLeoIdeam
Gold ⭐️
November 4, 2021
Solved

Oozie Java Action doesnt do anything

I am trying to use the oozie java action to run a hello world java program to test the functionality but no matter what I do it seems oozie does something and passed but does not run my java program

 

I confirmed this by entering garbage in the 'Main class' field and it still passes and doesn't do anything useful?

 

KeithLeoIdeam_0-1636055776688.png

 

This passes?? So what is oozie doing? and how do i get it to run the jar? (it is uploaded to hdfs)

 

KeithLeoIdeam_1-1636055830990.png

These are the logs which show no attempt at running my jar

 

Log Type: stdout
          
            Log Upload Time: Thu Nov 04 21:57:01 +0200 2021
          
            Log Length: 3809
          log4j: Trying to find [container-log4j.properties] using context classloader sun.misc.Launcher$AppClassLoader@5e2de80c.
log4j: Using URL [jar:file:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p48.16116877/jars/hadoop-yarn-server-nodemanager-3.1.1.7.1.4.48-1.jar!/container-log4j.properties] for automatic log4j configuration.
log4j: Reading configuration from URL jar:file:/opt/cloudera/parcels/CDH-7.1.4-1.cdh7.1.4.p48.16116877/jars/hadoop-yarn-server-nodemanager-3.1.1.7.1.4.48-1.jar!/container-log4j.properties
log4j: Hierarchy threshold set to [ALL].
log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
log4j: Level token is [INFO].
log4j: Category root set to INFO
log4j: Parsing appender named "CLA".
log4j: Parsing layout options for "CLA".
log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
log4j: End of parsing for "CLA".
log4j: Setting property [containerLogFile] to [syslog].
log4j: Setting property [totalLogFileSize] to [1048576].
log4j: Setting property [containerLogDir] to [/data/06/yarn/container-logs/application_1635807677896_29253/container_e33_1635807677896_29253_01_000001].
log4j: setFile called: /data/06/yarn/container-logs/application_1635807677896_29253/container_e33_1635807677896_29253_01_000001/syslog, true
log4j: setFile ended
log4j: Parsed "CLA" options.
log4j: Parsing appender named "EventCounter".
log4j: Parsed "EventCounter" options.
log4j: Parsing for [org.apache.hadoop.mapreduce.task.reduce] with value=[INFO,CLA].
log4j: Level token is [INFO].
log4j: Category org.apache.hadoop.mapreduce.task.reduce set to INFO
log4j: Parsing appender named "CLA".
log4j: Appender "CLA" was already parsed.
log4j: Handling log4j.additivity.org.apache.hadoop.mapreduce.task.reduce=[false]
log4j: Setting additivity for "org.apache.hadoop.mapreduce.task.reduce" to false
log4j: Parsing for [org.apache.hadoop.mapred.Merger] with value=[INFO,CLA].
log4j: Level token is [INFO].
log4j: Category org.apache.hadoop.mapred.Merger set to INFO
log4j: Parsing appender named "CLA".
log4j: Appender "CLA" was already parsed.
log4j: Handling log4j.additivity.org.apache.hadoop.mapred.Merger=[false]
log4j: Setting additivity for "org.apache.hadoop.mapred.Merger" to false
log4j: Finished configuring.
Launcher AM configuration loaded
Executing Oozie Launcher with tokens:
Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 29253 cluster_timestamp: 1635807677896 } attemptId: 1 } keyId: -581583983)
Kind: RM_DELEGATION_TOKEN, Service: 192.168.80.67:8032,192.168.80.68:8032, Ident: (owner=g983797, renewer=yarn, realUser=oozie/srv009066.mud.internal.co.za@ANDROMEDA.CLOUDERA, issueDate=1636055816551, maxDate=1636660616551, sequenceNumber=455083, masterKeyId=395)
Kind: HDFS_DELEGATION_TOKEN, Service: ha-hdfs:nameservice1, Ident: (token for g983797: owner=g983797, renewer=yarn, realUser=oozie/srv009066.mud.internal.co.za@ANDROMEDA.CLOUDERA, issueDate=1636055816526, maxDate=1636660616526, sequenceNumber=23258625, masterKeyId=1399)
Kind: MR_DELEGATION_TOKEN, Service: 192.168.80.68:10020, Ident: (owner=g983797, renewer=yarn, realUser=oozie/srv009066.mud.internal.co.za@ANDROMEDA.CLOUDERA, issueDate=1636055816543, maxDate=1636660616543, sequenceNumber=5124, masterKeyId=4)
Oozie Launcher, uploading action data to HDFS sequence file: hdfs://nameservice1/user/g983797/oozie-oozi/0485072-211028195417634-oozie-oozi-W/java-b4b4--java/action-data.seq
Stopping AM
Callback notification attempts left 0
Callback notification trying http://cloudera.sanlam.co.za:11000/oozie/callback?id=0485072-211028195417634-oozie-oozi-W@java-b4b4&status=SUCCEEDED
Callback notification to http://cloudera.sanlam.co.za:11000/oozie/callback?id=0485072-211028195417634-oozie-oozi-W@java-b4b4&status=SUCCEEDED succeeded
Callback notification succeeded

My java program does this for now

package za.co.sanlam;

/**
 * Hello world!
 *
 */
public class App 
{
    public static void main( String[] args )
    {
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );
        System.out.println( "Hello World!" );

    }
}
mszurap
mszurapAnswer
Platinum ⭐️
November 15, 2021

You've mentioned that even if you intentionally "break" the workflow with changing the class name it still does not throw an error message. Can you check what's in the workflow.xml when you change the workflow? This would help to decide is Hue not saving the workflow properly or is the logging broken only.

Is the workflow successful even with a wrong classname?

View in thread

4 replies

mszurap
Platinum ⭐️
mszurap
Platinum ⭐️
November 5, 2021

Hi,

Your approach seems ok, we need to figure out what went wrong.

After the "Executing Oozie Launcher with tokens" and the logging of the token retrievals you should see something like:

 

Executing Action Main with tokens:
...
Main class : org.apache.oozie.action.hadoop.JavaMain
...
Launcher class: class org.apache.oozie.action.hadoop.JavaMain
...
>>> Invoking Main class now >>>

Hello world!

<<< Invocation of Main class completed <<<

Do you observe such issues only with Java actions? Are the other action types working fine?

Can you check the workflow.xml through the following?

1. Open your workflow definition in Hue

2. At the right click the three-dotted icon > Workspace

3. Click the "workflow.xml" to open it

It should have an "<action name=...><java>" definiton with your latest edits in it.

 

Best regards

 Mike

Translate
KeithLeoIdeam
KeithLeoIdeamAuthor
Gold ⭐️
November 8, 2021

When I use shell or spark action things work as expected

 

this is the contents of the workflow.xml

 

<workflow-app name="Test Java Action" xmlns="uri:oozie:workflow:0.5">
    <start to="java-b4b4"/>
    <kill name="Kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <action name="java-b4b4">
        <java>
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <main-class>za.co.sanlam.App</main-class>
            <file>/user/g983797/test-1.0-SNAPSHOT.jar#test-1.0-SNAPSHOT.jar</file>
        </java>
        <ok to="End"/>
        <error to="Kill"/>
    </action>
    <end name="End"/>
</workflow-app>

 

yes agreed, that's what I expected, I expected to see some logging that would show the main class was invoked.

 

Given how simple the setup of the action is I am very confused about how to make it do anything else. It makes no sense what its actually doing or why its not running

Translate
mszurap
Platinum ⭐️
mszurapAnswer
Platinum ⭐️
November 15, 2021

You've mentioned that even if you intentionally "break" the workflow with changing the class name it still does not throw an error message. Can you check what's in the workflow.xml when you change the workflow? This would help to decide is Hue not saving the workflow properly or is the logging broken only.

Is the workflow successful even with a wrong classname?

Translate
Terms & ConditionsAccessibility statement