Cannot create a constant column in HCatalog

wgonzalez — Wed, 16 Dec 2015 07:05:29 GMT

HCatalog does not seem to allow Constants as columns.

Via Sqoop import:

sqoop import \
--query "select col1 as col1, col2 as col2, '' as col3, col4 as col4 from input_table" \
--connect jdbc:db2://server.com:60000/DB1 \
--username ******* \
--password ******* \
--hcatalog-database my_db \
--hcatalog-table output_table \
--create-hcatalog-table \
--hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")' \
--verbose \
--fetch-size 20000 \

Log:

15/12/14 03:28:57 INFO hcat.SqoopHCatUtilities: Creating HCatalog table my_db.output_table for import
15/12/14 03:28:57 INFO hcat.SqoopHCatUtilities: HCatalog Create table statement: 

create table `my_db`.`output_table` (
`col1` varchar(387),
`col2` varchar(17),
`col3` varchar, 
`col4` varchar(17))
stored as orc tblproperties ("orc.compress"="SNAPPY")
...
15/12/14 03:29:04 INFO hcat.SqoopHCatUtilities: FAILED: ParseException line 4:14 mismatched input ',' expecting ( near 'varchar' in primitive type specification
15/12/14 03:29:05 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@4aa0560e
15/12/14 03:29:05 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: HCat exited with status 64

Is this supported?

Looks like:

https://issues.apache.org/jira/browse/SQOOP-2596

Re: Cannot create a constant column in HCatalog

henrysowell — Wed, 16 Dec 2015 11:34:08 GMT

Sqoop is looking for the column precision for `col3`, and because there isn't a parenthesis and precision following 'varchar', it's throwing the error. The Jira you highlighted shows the same problem. Good news, they've written the patch for this and it should be coming in the next version of Sqoop.

Re: Cannot create a constant column in HCatalog

gbraccialli3 — Thu, 17 Dec 2015 22:42:56 GMT

@William Gonzalez

Maybe a cast can be a workaround, like this:

--query "select col1 as col1, col2 as col2, cast('' as varchar(10)) as col3, col4 as col4 from input_table" \

question Re: Cannot create a constant column in HCatalog in Archives of Support Questions (Read Only)

Cannot create a constant column in HCatalog

Re: Cannot create a constant column in HCatalog

Re: Cannot create a constant column in HCatalog