- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
In Sqoop import is there an option to overwrite or delete existing data in an hcatalog table
- Labels:
-
Apache HCatalog
-
Apache Sqoop
Created ‎10-16-2017 04:33 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
My requirement is to overwrite (or delete prior to the import) the existing data in an hcatalog table during sqoop import.
It appears hive-overwrite and delete-target-dir arguments don't work for this purpose.
Any suggestions on how to do this.
Thanks.
Created ‎10-26-2017 07:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you are using HCatalog with Sqoop, then Sqoop Hive options that are not supported with HCatalog jobs includes (link😞
- --hive-import
- --hive-overwrite
Approach is to use hive-import instead of -hcatalog. For example,
sqoop import --connect jdbc:mysql://localhost/test --username root --password 'hr' --table sample --hive-import --hive-overwrite --hive-table sqoophive -m 1 --fields-terminated-by '\t' --lines-terminated-by '\n'
Created ‎05-15-2018 03:59 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unfortunately "--hive-overwrite" option destroy hive table structure and re-create it after that which is not acceptable way.
The only way is:
1. hive> truncate table sample;
2. sqoop import --connect jdbc:mysql://yourhost/test --username test --password test01 --table sample --hcatalog-table sample
