Support Questions
Find answers, ask questions, and share your expertise

Using sqoop incremental import to a hive table is not supported ?

Solved Go to solution
Highlighted

Using sqoop incremental import to a hive table is not supported ?

Expert Contributor

Have been trying to do incremental import to a hive table using sqoop .. but unfortunately showing as Append mode for hive imports is not yet supported. Please remove the parameter --append-mode

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Using sqoop incremental import to a hive table is not supported ?

Hi @Krishna Srinivas, that's correct. See this for a workaround to do incremental import into a Hive external table by way of HDFS.

View solution in original post

5 REPLIES 5
Highlighted

Re: Using sqoop incremental import to a hive table is not supported ?

Hi @Krishna Srinivas, that's correct. See this for a workaround to do incremental import into a Hive external table by way of HDFS.

View solution in original post

Highlighted

Re: Using sqoop incremental import to a hive table is not supported ?

Expert Contributor

@Predrag Minovic seems fine but we are currently using ORC format of hive tables and for external managed tables is it possible for the same format ?

Highlighted

Re: Using sqoop incremental import to a hive table is not supported ?

Mentor

check this link

Was your code like below

sqoop import \

--connect jdbc:mysql://mysql.example.com/sqoop \

--username sqoop \

--password sqoop \

--table visits \

--incremental append \

--check-column id \

--last-value 1

16/03/18 08:16:36 INFO tool.ImportTool: Incremental import complete! ...

16/03/18 08:16:36 INFO tool.ImportTool: --incremental append

16/03/18 08:16:36 INFO tool.ImportTool: --check-column id

16/03/18 08:16:36 INFO tool.ImportTool: --last-value 2

Highlighted

Re: Using sqoop incremental import to a hive table is not supported ?

Expert Contributor

@Geoffrey Shelton Okot sqoop import --connect "jdbc:sqlserver://xxxxx:1433;database=xxxxxx;username=xxxxxx;password=xxxxx" --table <table> --hive-import --hive-database <DBname> --incremental append --check-column <name> --last-value 3 -m 1

Highlighted

Re: Using sqoop incremental import to a hive table is not supported ?

Expert Contributor
@Geoffrey Shelton Okot I am having issue while importing to Hive table and to HDFS it is happening perfectly fine