Sqooping data from LOCAL MSSQL Server to HDInsight over Azure?

[#1]

I am trying to Sqoop data from a local on-site Microsoft SQL server to Hive in HDinsights in Azure.

Inside Secure Shell in HDInsight, I CAN sqoop data in from Azure SQL Server but UNABLE to sqoop data from SQL Server locally.

I can also setup Azure SQL Server, setup firewall setting and synchronize with local SQL Server. I cannot connect/ping the external SQL Server's ip.

- why can't my sqoop connect to the local SQL Server?

- are there any firewalls setup in HDInsight that prevents connection?

[#2]

@rmian

If endpoints are set for the ports then connectivity should not be an issue.

[#3]

It looks like network issue rather than Sqoop issue as you are mentioning you are not able to ping external SQL Server machine.

[#4]

@Pardeep

Strange...able to setup Azure SQL Server, setup firewall setting and synchronize with local SQL Server. Possibly something specific to HDInsight....

[#5]

Give Azure Data Factory (ADF) a try.

[#6 · Accepted solution]

@pardeep

answer from microsoft: "I believe this will not work unless your HDInsight cluster and your on premise SQL server are connected thorugh a VPN.

I would recommend trying alternatuves like Azure Data Factory for such a hybrid data movement scenario."

https://social.msdn.microsoft.com/Forums/azure/en-US/36646ef9-221e-4e8e-b80e-c91316857958/sqooping-d...