- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Sqoop - specifying file name when importing data
- Labels:
-
Apache Sqoop
Created on 06-19-2014 07:00 AM - edited 09-16-2022 02:00 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is there a way to set a file name in which data is beeing imported? File name is by default part-m* (or something like that).
Created 07-26-2014 08:17 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Using Sqoop1, you should be able to provide --append to add to an existing directory. See http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_controlling_the_import_process for more info.
Created 06-28-2014 02:16 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Unfortunately, that would require a custom output format, which is currently not supported in Sqoop 1. You might be able to get away with writing a custom storage format for HCatalog.
This seems like an interesting feature to add in Sqoop2 though: https://issues.apache.org/jira/browse/SQOOP-1346.
Created 07-25-2014 02:31 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Ok, but Is there a way to gather data from several servers in a loop with sqoop import, in the same directory, since we can't specify a file name? Each server would return file with the same name?
Created 07-26-2014 08:17 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Using Sqoop1, you should be able to provide --append to add to an existing directory. See http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_controlling_the_import_process for more info.
