question Re: Using sqoop incremental import to a hive table is not supported ? in Archives of Support Questions (Read Only)

Using sqoop incremental import to a hive table is not supported ?

klksrinivas — Tue, 09 Feb 2016 18:18:46 GMT

Have been trying to do incremental import to a hive table using sqoop .. but unfortunately showing as Append mode for hive imports is not yet supported. Please remove the parameter --append-mode

Re: Using sqoop incremental import to a hive table is not supported ?

pminovic — Tue, 09 Feb 2016 18:25:55 GMT

Hi @Krishna Srinivas, that's correct. See this for a workaround to do incremental import into a Hive external table by way of HDFS.

Re: Using sqoop incremental import to a hive table is not supported ?

Shelton — Tue, 09 Feb 2016 18:36:30 GMT

check this link

Was your code like below

sqoop import \

--connect jdbc:mysql://mysql.example.com/sqoop \

--username sqoop \

--password sqoop \

--table visits \

--incremental append \

--check-column id \

--last-value 1

16/03/18 08:16:36 INFO tool.ImportTool: Incremental import complete! ...

16/03/18 08:16:36 INFO tool.ImportTool: --incremental append

16/03/18 08:16:36 INFO tool.ImportTool: --check-column id

16/03/18 08:16:36 INFO tool.ImportTool: --last-value 2

Re: Using sqoop incremental import to a hive table is not supported ?

klksrinivas — Tue, 09 Feb 2016 19:42:27 GMT

@Geoffrey Shelton Okot sqoop import --connect "jdbc:sqlserver://xxxxx:1433;database=xxxxxx;username=xxxxxx;password=xxxxx" --table <table> --hive-import --hive-database <DBname> --incremental append --check-column <name> --last-value 3 -m 1

Re: Using sqoop incremental import to a hive table is not supported ?

klksrinivas — Tue, 09 Feb 2016 19:43:14 GMT

@Geoffrey Shelton Okot I am having issue while importing to Hive table and to HDFS it is happening perfectly fine

Re: Using sqoop incremental import to a hive table is not supported ?

klksrinivas — Tue, 09 Feb 2016 20:26:57 GMT

@Predrag Minovic seems fine but we are currently using ORC format of hive tables and for external managed tables is it possible for the same format ?