Reply
Highlighted
Contributor
Posts: 43
Registered: ‎08-20-2013

Oozie sqoop action with "eval" looking for work-around.

I have an oozie workflow that truncates a mysql table and then exports from hdfs into this table, 

 

The truncate using "sqoop eval" returns error similar to the following: 

http://hortonworks.com/community/forums/topic/oozie-via-hue-sqoop-eval-failing/

 

Here is running CDH4.4.0 with 

SqoopCDH41.4.3+81
Sqoop2 (CDH4 only)CDH41.99.2+98

 

I don't want to compile from trunk, what is my work-around to truncate the table before exporting to it?

Cloudera Employee
Posts: 35
Registered: ‎07-08-2013

Re: Oozie sqoop action with "eval" looking for work-around.

I don't think that's the same problem described on that other forum; they said that it was OOZIE-1447 that fixed it.  However, CDH 4.4.0 has OOZIE-1447 already.  

 

Can you post the error?  Also keep in mind that using eval isn't recommended/supported.

Software Engineer | Cloudera, Inc. | http://cloudera.com
Contributor
Posts: 43
Registered: ‎08-20-2013

Re: Oozie sqoop action with "eval" looking for work-around.

Here is the sqoop command: 

 

eval --connect jdbc:mysql://xxxxx/reporting -e "truncate table apiusage" --username sqoop --password xxxxx

 

Here is the error: 

 

>>> Invoking Sqoop command line now >>>

1292 [main] WARN  org.apache.sqoop.tool.SqoopTool  - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
1327 [main] INFO  org.apache.sqoop.Sqoop  - Running Sqoop version: 1.4.3-cdh4.5.0
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Error parsing arguments for eval:
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: table
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: apiusage"
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: --username
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: sqoop
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: --password
1342 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool  - Unrecognized argument: sqoop
usage: sqoop eval [GENERIC-ARGS] [TOOL-ARGS]

Common arguments:
--connect <jdbc-uri>                         Specify JDBC connect
string
--connection-manager <class-name>            Specify connection manager
class name
--connection-param-file <properties-file>    Specify connection
parameters file
--driver <class-name>                        Manually specify JDBC
driver class to use
--hadoop-home <hdir>                         Override
$HADOOP_MAPRED_HOME_ARG
--hadoop-mapred-home <dir>                   Override
$HADOOP_MAPRED_HOME_ARG
--help                                       Print usage instructions
-P                                              Read password from console
--password <password>                        Set authentication
password
--password-file <password-file>              Set authentication
password file path
--skip-dist-cache                            Skip copying jars to
distributed cache
--username <username>                        Set authentication
username
--verbose                                    Print more information
while working

SQL evaluation arguments:
-e,--query <statement>    Execute 'statement' in SQL and exit

Generic Hadoop command-line arguments:
(must preceed any tool-specific arguments)
Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|jobtracker:port>    specify a job tracker
-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.

The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]

Intercepting System.exit(1)

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

 

Contributor
Posts: 43
Registered: ‎08-20-2013

Re: Oozie sqoop action with "eval" looking for work-around.

Error attached. Space in the Sql statement seems being parsed out as separate items.
Contributor
Posts: 43
Registered: ‎08-20-2013

Re: Oozie sqoop action with "eval" looking for work-around.

If this is not related to oozie-1447, then must be a new bug. Can someone investigate? Thanks.
Cloudera Employee
Posts: 9
Registered: ‎08-15-2013

Re: Oozie sqoop action with "eval" looking for work-around.

As mentioned before the eval function is for evaluation and not meant for production use at this time.  Can you write a simple Java action to do the truncate?

Expert Contributor
Posts: 109
Registered: ‎05-19-2016

Re: Oozie sqoop action with "eval" looking for work-around.

Did you find a way?

Announcements