Support Questions

Find answers, ask questions, and share your expertise

Sqoop - specifying file name when importing data

avatar
Explorer

Is there a way to set a file name in which data is beeing imported? File name is by default part-m* (or something like that).

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor
Actually, it looks like you might be able to change the base filename using "mapreduce.output.basename". Try adding -Dmapreduce.output.basename=myoutputprefix or adding this to the sqoop-site.xml?

Using Sqoop1, you should be able to provide --append to add to an existing directory. See http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_controlling_the_import_process for more info.

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

Unfortunately, that would require a custom output format, which is currently not supported in Sqoop 1. You might be able to get away with writing a custom storage format for HCatalog.

 

This seems like an interesting feature to add in Sqoop2 though: https://issues.apache.org/jira/browse/SQOOP-1346.

avatar
Explorer

Ok, but Is there a way to gather data from several servers in a loop with sqoop import, in the same directory, since we can't specify a file name? Each server would return file with the same name?

avatar
Expert Contributor
Actually, it looks like you might be able to change the base filename using "mapreduce.output.basename". Try adding -Dmapreduce.output.basename=myoutputprefix or adding this to the sqoop-site.xml?

Using Sqoop1, you should be able to provide --append to add to an existing directory. See http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_controlling_the_import_process for more info.