Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Does SQOOP support bulk inserts into MS SQL Server?

Does SQOOP support bulk inserts into MS SQL Server?

Explorer

I have a requirement to SQOOP out almost 1TB of data out from Cluster to SQL server. I am finding it time consuming to SQOOP out the  same using sqlJDBC4.jar driver. IT tool me something like 12hrs for 137GB of data. I am looking for some bulk insert options. Would appreciate if any one can provide pointers to the same.

 

Thanks,

Vinayak(Vin).

3 REPLIES 3

Re: Does SQOOP support bulk inserts into MS SQL Server?

New Contributor
Did you ever find an answer to this question? I can see that there are newer versions of the JDBC now.

Re: Does SQOOP support bulk inserts into MS SQL Server?

Explorer
Not yet.

Re: Does SQOOP support bulk inserts into MS SQL Server?

Cloudera Employee

Hello Vinshin,

 

The Sqoop1 CLI supports the "--batch" option with translates into JDBC addBatch().  The implementation of addBatch (if and/or how) is completely up to the RDBMS vendor, Sqoop1 really does not have any control over it other than the ability to turn it on/off with --batch.

 

Table 29. Export control arguments:

--batch Use batch mode for underlying statement execution.

http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.5.0/SqoopUserGuide.html#_syntax_4

 

Sqoop1 does support MS SQL Server direct options however there does not seem to be anything specific to bulk load.

 

25.3. Microsoft SQL Connector

http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.5.0/SqoopUserGuide.html#_microsoft_sql_conne...

 

Is there a specific option that you are looking for that you could provide a MS SQL Server documentation reference to that we could review further?

 

Thank you,

Markus Kemper

Cloudera Support 

Don't have an account?
Coming from Hortonworks? Activate your account here