Running Sqoop via Oozie only gives error: "[org.apache.oozie.action.hadoop.SqoopMain], exit code [1]", empty stdout

frank_wein — Tue, 10 Apr 2018 21:44:01 GMT

Hi all,

I'm trying to run Sqoop via Oozie as a first test workflow (from the Workflow Manager view in HDP). I've setup my own installation of the current HDP version. However, my workflow always fails to execute, Oozie tells me:

Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1], External status: FAILED/KILLED

I've looked at the instructions at https://community.hortonworks.com/articles/9148/troubleshooting-an-oozie-flow.html to find the real error message, but the stdout log is empty:

Log Type: stdout
Log Upload Time: Tue Apr 10 16:29:39 +0200 2018
Log Length: 0

and stderr does not give me any useful information as far as I see this:

Apr 10, 2018 4:30:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Apr 10, 2018 4:30:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Apr 10, 2018 4:30:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Apr 10, 2018 4:30:04 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
Apr 10, 2018 4:30:04 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
Apr 10, 2018 4:30:06 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton"
Apr 10, 2018 4:30:09 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope "PerRequest"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

Does someone have a good idea what else I could try to get Sqoop running via Oozie? I've looked at quite a few entries here in the community, but found nothing promising so far. Executing Sqoop locally in the shell works fine after fixing the JDBC drivers.

Re: Running Sqoop via Oozie only gives error: "[org.apache.oozie.action.hadoop.SqoopMain], exit code [1]", empty stdout

frank_wein — Tue, 10 Apr 2018 22:09:16 GMT

Actually I think I manually found the logs for stdout now: I copied the application id from the Hadoop management website and executed the command

yarn logs -applicationId [applicationId]

on command line and there I found what I was looking for (a better log :). For the part I was looking for it said below the log:

End of LogType:stdout
***********************************************************************

Container: container_e03_1523366722691_0006_01_000002 on ubuntu_45454
LogAggregationType: AGGREGATED

Can I find this type of log not in the web interface then? Or do I need to check in a different web interface?

Edit: Looks like Hadoop Web interface only knew about attempt Nr. 1 in the job history for the MapReduce job (see the 000002 in the excerpt above). When I manually change the URL of the container ID to 00002 (e.g. ... container_e03_1523366722691_0006_01_000002/job_1523366722691_0006/admin), I can also see the log in the web interface. Confusing 🙂 anyway, it looks like I used the wrong arguments somehow for Sqoop, even though I'm quite sure it worked this way in the console. Problem solved (for now).

question Running Sqoop via Oozie only gives error: "[org.apache.oozie.action.hadoop.SqoopMain], exit code [1]", empty stdout in Archives of Support Questions (Read Only)

Running Sqoop via Oozie only gives error: "[org.apache.oozie.action.hadoop.SqoopMain], exit code [1]", empty stdout

Re: Running Sqoop via Oozie only gives error: "[org.apache.oozie.action.hadoop.SqoopMain], exit code [1]", empty stdout