question Sqoop performance regarding input format in Archives of Support Questions (Read Only)

Sqoop performance regarding input format

pvillard — Thu, 09 Jun 2016 18:24:24 GMT

In the case I want to export data, using Sqoop, from HDFS to an external destination (Teradata for example), is there a recommendation regarding the format of the input files?

AFAIK, supported formats are :

Delimited text files
Sequence files
ORC files

Do we observe performance differences between input formats?

Thanks

Re: Sqoop performance regarding input format

ssubhas — Thu, 09 Jun 2016 18:40:51 GMT

@Pierre Villard

Sqoop internally using yarn jobs for extracting data from HDFS. ORC is regarding as better performance for read even with Hive: You can refer to below link for details:

http://www.slideshare.net/StampedeCon/choosing-an-hdfs-data-storage-format-avro-vs-parquet-and-more-stampedecon-2015

Hope this helps.

Thanks and Regards,

Sindhu