Support Questions

Find answers, ask questions, and share your expertise

Cannot create a constant column in HCatalog

avatar

HCatalog does not seem to allow Constants as columns.

Via Sqoop import:

sqoop import \
--query "select col1 as col1, col2 as col2, '' as col3, col4 as col4 from input_table" \
--connect jdbc:db2://server.com:60000/DB1 \
--username ******* \
--password ******* \
--hcatalog-database my_db \
--hcatalog-table output_table \
--create-hcatalog-table \
--hcatalog-storage-stanza 'stored as orc tblproperties ("orc.compress"="SNAPPY")' \
--verbose \
--fetch-size 20000 \

Log:

15/12/14 03:28:57 INFO hcat.SqoopHCatUtilities: Creating HCatalog table my_db.output_table for import
15/12/14 03:28:57 INFO hcat.SqoopHCatUtilities: HCatalog Create table statement: 

create table `my_db`.`output_table` (
`col1` varchar(387),
`col2` varchar(17),
`col3` varchar, 
`col4` varchar(17))
stored as orc tblproperties ("orc.compress"="SNAPPY")
...
15/12/14 03:29:04 INFO hcat.SqoopHCatUtilities: FAILED: ParseException line 4:14 mismatched input ',' expecting ( near 'varchar' in primitive type specification
15/12/14 03:29:05 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@4aa0560e
15/12/14 03:29:05 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: HCat exited with status 64 

Is this supported?

Looks like:

https://issues.apache.org/jira/browse/SQOOP-2596

1 ACCEPTED SOLUTION

avatar
Contributor

Sqoop is looking for the column precision for `col3`, and because there isn't a parenthesis and precision following 'varchar', it's throwing the error. The Jira you highlighted shows the same problem. Good news, they've written the patch for this and it should be coming in the next version of Sqoop.

View solution in original post

2 REPLIES 2

avatar
Contributor

Sqoop is looking for the column precision for `col3`, and because there isn't a parenthesis and precision following 'varchar', it's throwing the error. The Jira you highlighted shows the same problem. Good news, they've written the patch for this and it should be coming in the next version of Sqoop.

avatar

@William Gonzalez

Maybe a cast can be a workaround, like this:

--query "select col1 as col1, col2 as col2, cast('' as varchar(10)) as col3, col4 as col4 from input_table" \