06-28-2017 08:13 AM
chd5.11.0 sqoop 1.4.6
Using Hue to create Workflow with Sqoop 1 works great for pulling entire table from MSSQL Server but fails for tables that are not in the default (dbo) schema.
Adding --schema schema_name_here anywhere in the sqoop command results in an "Error parsing arguments for import:" with an "Unrecognized argument: --schema".
Here is my sqoop command as entered in Hue:
import --connect jdbc:sqlserver://MSSQLP001;database=Cust --username ******* --password ******* --table Customer_Address --delete-target-dir --target-dir hdfs://cdh.dom:8020/user/sqlget/Cust/Incoming/Raw/Customer_Address -m 1 --schema customer
Here is the error it produces in the logs:
2017-06-28 09:21:26,416 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. 2017-06-28 09:21:26,465 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.6-cdh5.11.0 2017-06-28 09:21:26,481 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your password on the command-line is insecure. Consider using -P instead. 2017-06-28 09:21:26,482 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing arguments for import: 2017-06-28 09:21:26,482 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: --schema 2017-06-28 09:21:26,482 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: customer
<<< Invocation of Sqoop command completed <<<
I have tried moving the --schema argument in each possible location between the other arguments (connect and username, username and password, password and table, table and delete-target-dir, delete-target-dir and target-dir, target-dir and m, and as shown as the last command. The only difference this makes is all arguments after the --schema are unrecognized.
Apache shows v1.4.6 supports this argument in its Sqoop User Guide. Is the --schema argument not supported within the Sqoop 1 command.
I have successfully used this to pull dozens of tables but it only works for tables in the default (dbo) schema of the MS SQL Server database.
If there is any Cloudera documentation, examples, or other information on how to use Hue Workflows (Oozie) I would really appreciate it. I have had to resort to trial and error and brute force methods trying every possible permutation so having realy documentation or examples would be really helpful.