- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
how to load the data from sql server to hdfs using sqoop?
- Labels:
-
Apache Sqoop
-
HDFS
Created on 02-13-2017 02:52 PM - edited 09-16-2022 04:04 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 02-13-2017 02:52 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have often used this article:
Syntax: sqoop import --connect jdbc:sqlserver://sqlserver-name \ --username <username> \ --password <password> \ --driver <driver-manager-class> \ --table <table-name> \ --target-dir <target-folder-name> Sample: sqoop import --connect "jdbc:sqlserver://labDB;database=demo" \ --username sqoop \ --password simp1e \ --driver com.microsoft.sqlserver.jdbc.SQLServerDriver \ --table "dbo.customer" \ --target-dir "/tmp/dbo-customer"
Created 02-13-2017 02:52 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Just add the SQL Server jdbc jar(sqljdbc42.jar) to the sqoop lib folder and try to sqoop using the syntax provided above bu Sunile.
Created 06-20-2017 04:45 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am getting the error regarding specifying connection manager even though I have added the jar file at /usr/hdp/current/sqoop_client/lib
Can anyone help?
I have used the same command as given above.
Error-
WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time. 17/06/20 11:28:48 INFO manager.SqlManager: Using default fetchSize of 1000 17/06/20 11:28:48 INFO tool.CodeGenTool: Beginning code generation 17/06/20 11:28:48 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.microsoft.jdbc.sqlserver.SQLServerDriver
Created on 03-16-2017 07:01 AM - edited 08-19-2019 04:47 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 03-16-2017 07:48 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Harold Allen Badilla In your command, you are using mysql database connection string i.e "--connect jdbc:mysql://XXXXX" with "--driver" option with value as SQL Server driver class name. So your connection string should be changed as SQL Server format.
Proper command syntax will look something like this,
sqoop import --connect --connect "jdbc:sqlserver://<Server_Host>:<Server_Port>;databaseName=<DB_Name>" \ --driver com.microsoft.sqlserver.jdbc.SQLServerDriver \ --username XXXXXXXXXXXX --password XXXXXXXXXXX \ --table dbo.knal \ --target-dir /tmp/db0-customer
Created on 03-17-2017 01:08 AM - edited 08-19-2019 04:47 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I try to revise my commands , i dont see any error but still no data transferred on the hdfs directory, please the image
Created 03-17-2017 05:34 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@Harold Allen BadillaDid you add the sqljdbc42.jar to sqoop lib path?, if not please add the jar.
And driver class name as "com.microsoft.sqlserver.jdbc.SQLServerDriver".
Created 06-20-2017 01:05 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Created 03-17-2017 04:36 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content