Support Questions
Find answers, ask questions, and share your expertise

Is Sqoop robust over WAN for a production HDInsight cluster?

Is Sqoop robust over WAN for a production HDInsight cluster?

Explorer

Need to pull data from MSSQL over WAN into Hive. Storing them as files on HDFS would be a bonus.

Curios to know if Sqoop is able to provide a robust import over WAN? Would Sqoop meet requirements of a production environment?

Thanking in advance

2 REPLIES 2
Highlighted

Re: Is Sqoop robust over WAN for a production HDInsight cluster?

@rmian

Sqoop is able to provide a robust import over WAN ? Yes but exact answer based on the setup

- Connection speed/bandwidth

- Security "Very Important"

Would Sqoop meet requirements of a production environment? Yes http://hortonworks.com/hadoop/sqoop/

Highlighted

Re: Is Sqoop robust over WAN for a production HDInsight cluster?

New Contributor

If you have other data processing occuring inside Azure take a look at Azure Data Factory for your data movement. It can orchestrate reads and writes across SQL Server, Hive, Pig, and much more.