Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Oozie launcher never ends

Oozie launcher never ends

Explorer

Hello,

I'm doing a simple oozie's workflow with sqqop's action.
this is the xml's code

 

<workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4">
    <start to="Sqoop"/>
    <action name="Sqoop">
        <sqoop xmlns="uri:oozie:sqoop-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <command>import --connect jdbc:mysql://10.0.0.221/db --username cloudera --password cloudera --table test -m 1 --target-dir /user/albert</command>
        </sqoop>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

When i launch my job with hue or line-command, starts the oozie-launcher.
My oozie laucher, never finish, it is always stay at 95%.
If i kill the job oozie-launcher, starts the oozie-sqoop and it do his work very good.
But i don't understand why the oozie-launcher doesnt' work well!

 

This is my output

 

>>> Invoking Sqoop command line now >>>

12280 [main] WARN  org.apache.sqoop.tool.SqoopTool  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
12510 [main] INFO  org.apache.sqoop.Sqoop  - Running Sqoop version: 1.4.4-cdh5.0.1
12640 [main] WARN  org.apache.sqoop.tool.BaseSqoopTool  - Setting your password on the command-line is insecure. Consider using -P instead.
12748 [main] WARN  org.apache.sqoop.ConnFactory  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
13131 [main] INFO  org.apache.sqoop.manager.MySQLManager  - Preparing to use a MySQL streaming resultset.
13131 [main] INFO  org.apache.sqoop.tool.CodeGenTool  - Beginning code generation
15107 [main] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
15202 [main] INFO  org.apache.sqoop.manager.SqlManager  - Executing SQL statement: SELECT t.* FROM `test` AS t LIMIT 1
15206 [main] INFO  org.apache.sqoop.orm.CompilationManager  - HADOOP_MAPRED_HOME is /opt/cloudera/parcels/CDH-5.0.1-1.cdh5.0.1.p0.47/lib/hadoop-mapreduce
23112 [main] INFO  org.apache.sqoop.orm.CompilationManager  - Writing jar file: /tmp/sqoop-yarn/compile/ab1d242ba1b3a0869bab06c1eb20c02f/test.jar
23140 [main] WARN  org.apache.sqoop.manager.MySQLManager  - It looks like you are importing from mysql.
23140 [main] WARN  org.apache.sqoop.manager.MySQLManager  - This transfer can be faster! Use the --direct
23140 [main] WARN  org.apache.sqoop.manager.MySQLManager  - option to exercise a MySQL-specific fast path.
23140 [main] INFO  org.apache.sqoop.manager.MySQLManager  - Setting zero DATETIME behavior to convertToNull (mysql)
23151 [main] INFO  org.apache.sqoop.mapreduce.ImportJobBase  - Beginning import of test
23232 [main] WARN  org.apache.sqoop.mapreduce.JobBase  - SQOOP_HOME is unset. May not be able to find all job dependencies.
25731 [main] INFO  org.apache.sqoop.mapreduce.db.DBInputFormat  - Using read commited transaction isolation
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat

 

 

Someone can help me?

Thank-you

16 REPLIES 16

Re: Oozie launcher never ends

Contributor

Hi,

 

The launcher job sticks around the entire time the sqoop job is running (because it's running the sqoop CLI).  So when using Oozie, you need to make sure that your cluster has capacity.  In MR1 this was quite common if you didn't have enough map slots.  In MR2, I've seen this happen if you haven't configured your Scheduler properly or if you don't have enough memory in your node manager.

Software Engineer | Cloudera, Inc. | http://cloudera.com

Re: Oozie launcher never ends

Explorer

I think could be right idea!
I change on cloudera manager max running apps for user root in cluster->Dynamic resource pools.
i also change in YARN configuration the java-heap-space.

Is there something else to change, may be in oozie configuration?

Re: Oozie launcher never ends

Explorer

I'm trying this workflow under AWS-ec2 machines, for testing.
I have changed my mini-cluster with 3 nodes, with machines powerful (x.medium -> x2.large) and my workflow it's ok.
I search on internet some impo, but i don't find a tutorial-rules, there is some link for this? now i set virtual core, and other impostation with my mind but i want some more efficient.

Re: Oozie launcher never ends

New Contributor
Was there ever resolution to this ? I am having the same problem.

Re: Oozie launcher never ends

Explorer

Hi,

 

I changed the power of the machines in aws.
I have also some parameter of yarn, but the real change was to have more powerful machines.

Re: Oozie launcher never ends

Master Guru
The problem was most likely that the NodeManager's exposed container and cpu container configs were set too low by default in the YARN configuration. Increasing this to appropriate values, such as ones suggested by http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cdh_ig_mapreduce_to_yar... will most likely resolve your 'hanging jobs' issue.

Re: Oozie launcher never ends

Explorer
hi Harsh J ,
I Have same Probleme, so i work with cloudera quickstart 5.10 .
the machine have 48 GO Ram , 4 core CPU.
but the problem stiil , the log show me "Heart beat "
any solution plz !!
thank you !!

Re: Oozie launcher never ends

Explorer

How many vCPU are allocated to yarn ?

If the answer is 4 or less, then you don't have enough vCPU allocated to yarn for this to work.

Re: Oozie launcher never ends

Explorer

Hi Math,

i allocated 3 vCPU max to yarn !

so, should  it allocate 5 vCPU to Hardware Machine and yarn also !!

Note : i work with quickstart cloudera 5.10 machine !

 

think you !