Created 12-15-2015 11:05 PM
HCatalog does not seem to allow Constants as columns.
Via Sqoop import:
sqoop import \ --query "select col1 as col1, col2 as col2, '' as col3, col4 as col4 from input_table" \ --connect jdbc:db2://server.com:60000/DB1 \ --username ******* \ --password ******* \ --hcatalog-database my_db \ --hcatalog-table output_table \ --create-hcatalog-table \ --hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")' \ --verbose \ --fetch-size 20000 \
Log:
15/12/14 03:28:57 INFO hcat.SqoopHCatUtilities: Creating HCatalog table my_db.output_table for import 15/12/14 03:28:57 INFO hcat.SqoopHCatUtilities: HCatalog Create table statement: create table `my_db`.`output_table` ( `col1` varchar(387), `col2` varchar(17), `col3` varchar, `col4` varchar(17)) stored as orc tblproperties ("orc.compress"="SNAPPY") ... 15/12/14 03:29:04 INFO hcat.SqoopHCatUtilities: FAILED: ParseException line 4:14 mismatched input ',' expecting ( near 'varchar' in primitive type specification 15/12/14 03:29:05 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@4aa0560e 15/12/14 03:29:05 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: HCat exited with status 64
Is this supported?
Looks like:
Created 12-16-2015 03:34 AM
Sqoop is looking for the column precision for `col3`, and because there isn't a parenthesis and precision following 'varchar', it's throwing the error. The Jira you highlighted shows the same problem. Good news, they've written the patch for this and it should be coming in the next version of Sqoop.
Created 12-16-2015 03:34 AM
Sqoop is looking for the column precision for `col3`, and because there isn't a parenthesis and precision following 'varchar', it's throwing the error. The Jira you highlighted shows the same problem. Good news, they've written the patch for this and it should be coming in the next version of Sqoop.
Created 12-17-2015 02:42 PM
Maybe a cast can be a workaround, like this:
--query "select col1 as col1, col2 as col2, cast('' as varchar(10)) as col3, col4 as col4 from input_table" \