question Re: Sqoop - specifying file name when importing data in Archives of Support Questions (Read Only)

Sqoop - specifying file name when importing data

MarsaPalas — Fri, 16 Sep 2022 09:00:31 GMT

Is there a way to set a file name in which data is beeing imported? File name is by default part-m* (or something like that).

Re: Sqoop - specifying file name when importing data

abe — Sat, 28 Jun 2014 21:16:43 GMT

Unfortunately, that would require a custom output format, which is currently not supported in Sqoop 1. You might be able to get away with writing a custom storage format for HCatalog.

This seems like an interesting feature to add in Sqoop2 though: https://issues.apache.org/jira/browse/SQOOP-1346.

Re: Sqoop - specifying file name when importing data

MarsaPalas — Fri, 25 Jul 2014 21:31:02 GMT

Ok, but Is there a way to gather data from several servers in a loop with sqoop import, in the same directory, since we can't specify a file name? Each server would return file with the same name?

Re: Sqoop - specifying file name when importing data

abe — Sat, 26 Jul 2014 15:17:41 GMT

Actually, it looks like you might be able to change the base filename using "mapreduce.output.basename". Try adding -Dmapreduce.output.basename=myoutputprefix or adding this to the sqoop-site.xml?

Using Sqoop1, you should be able to provide --append to add to an existing directory. See http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_controlling_the_import_process for more info.