Support Questions
Find answers, ask questions, and share your expertise

fastest way to ingest netezza tables into hive

Expert Contributor

Hi all,

i am using sqoop --direct mode now to copy nz tables into hive. If i run 4 parallel sqoop jobs, how does it affect hdp cluster and also how does it effect nz database?

also, is there a better and fastest way to import data from nz?


Re: fastest way to ingest netezza tables into hive



You can increase or decrese the number of mappers used in sqoop command using -m option. This helps you determine the number of parallel connections to be made to netezza. Though, please note, this doe snot always works, sometime the number of mappers are still decided on the basis of splits.

The impact of this would be on Yarn, the number of mappers would add load on Yarn as it will run as a Map-Reduce job.