Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Using sqoop incremental import to a hive table is not supported ?

avatar
Super Collaborator

Have been trying to do incremental import to a hive table using sqoop .. but unfortunately showing as Append mode for hive imports is not yet supported. Please remove the parameter --append-mode

1 ACCEPTED SOLUTION

avatar
Master Guru

Hi @Krishna Srinivas, that's correct. See this for a workaround to do incremental import into a Hive external table by way of HDFS.

View solution in original post

5 REPLIES 5

avatar
Master Guru

Hi @Krishna Srinivas, that's correct. See this for a workaround to do incremental import into a Hive external table by way of HDFS.

avatar
Super Collaborator

@Predrag Minovic seems fine but we are currently using ORC format of hive tables and for external managed tables is it possible for the same format ?

avatar
Master Mentor

check this link

Was your code like below

sqoop import \

--connect jdbc:mysql://mysql.example.com/sqoop \

--username sqoop \

--password sqoop \

--table visits \

--incremental append \

--check-column id \

--last-value 1

16/03/18 08:16:36 INFO tool.ImportTool: Incremental import complete! ...

16/03/18 08:16:36 INFO tool.ImportTool: --incremental append

16/03/18 08:16:36 INFO tool.ImportTool: --check-column id

16/03/18 08:16:36 INFO tool.ImportTool: --last-value 2

avatar
Super Collaborator

@Geoffrey Shelton Okot sqoop import --connect "jdbc:sqlserver://xxxxx:1433;database=xxxxxx;username=xxxxxx;password=xxxxx" --table <table> --hive-import --hive-database <DBname> --incremental append --check-column <name> --last-value 3 -m 1

avatar
Super Collaborator
@Geoffrey Shelton Okot I am having issue while importing to Hive table and to HDFS it is happening perfectly fine