Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Sqoop cannot load a driver class, SQL Server, when creating a connection

avatar
Explorer

I'm using CDH5.0.2(automated install). Single node hadoop installation. CDH manager is installed on a different node.

I follow this guide to install JDBC driver for SQL server. 

after copying sqljdbc4.jar to /var/lib/sqoop2, i restarted both sqoop2 and hue service. but when i create a new sqoop transfer job using hue's web ui, the jdbc driver class can't find any sql server driver. When i typed in "sql" in the jdbc driver class, it can only find MySql and PostGres Sql driver. it cannot find SQL server driver.

 

I tried changing the owenership(to sqoop2:sqoop2) and also the permission to the sqljdbc4.jar(to 644) to match that is of postgresql-9.0-801.jdbc4.jar in /var/lib/sqoop2, and restart the sqoop2 and hue service again, and still no luck. It still can't seem to find SQL server driver when i typed in SQL. What should be the value of this field for MS SQL Server??

 

I would really appreciate any help. I've ben evaluating CDH5, and can't seem to figure out this problem. 

 

1 ACCEPTED SOLUTION

avatar
Expert Contributor
It seems the issue is that your partition column can't be found. Sqoop should, by default, use the primary key as your partition column. If your table doesn't have a primary key, then you'll have to set it your self in "Partition column name".

View solution in original post

19 REPLIES 19

avatar
Explorer

Bump!

 

Same problem here baj - haven't found any documentation that correctly speaks to a CDH5.02 install.  We're going to need some help here please!

avatar
Expert Contributor

Hey guys,

 

Just a few questions to help me understand what you're seeing:

  1. Could you provide the exact exception you're seeing?
  2. Also, what connection string are you using?
  3. Is this with Parcels or Packages?

-Abe

avatar
Explorer

in my case, i'm using parcel, i'm not getting any exception. everything seems to install correctly(extracting jand copying sqljdbc4.jar to /var/lib/sqoop2).

 

I restarted both sqoop2 and hue service. but when i create a new sqoop transfer job using hue's web ui, the jdbc driver class can't find any sql server driver. When i typed in "sql" in the jdbc driver class, it can only find MySql and PostGres Sql driver. it cannot find SQL server driver.

 

I tried using sqoop1 via command line, it works, but not in sqoop2. 

avatar
Expert Contributor

Ah are you referring to the Hue UI? You should be able to add the SQL Server JDBC driver even though it's not part of the autocompletion list. That's local to Hue. I believe this is: com.microsoft.sqlserver.jdbc.SQLServerDriver.

avatar
Explorer

Hi Abe,

 

Thanks for your help!  My question is around installation - what directory should I put the sqljdbc4.jar file in?  This is a stock Cloudera install (parcels) done just last week on a 3-node CentOS cluster.  It appears most things are installed under:  /opt/cloudera/parcels/CDH-5.0.1-1.cdh5.0.1.p0.47, but I've found other bits and pieces elsewhere.  Or even better, and I do this from Cloudera Cluster Manager?

 

When I use HUE/Scoop2 UI and attempt to define a new connection, I get this error on the JDBC Driver Class field: Can't load specified driver.  The value I put in this field is "com.microsoft.sqlserver.jdbc.SQLServerDriver".

 

I've put sqljdbc4.jar in the /opt/cloudera/parcels/CDH/lib/sqoop/lib directory.

 

Many thanks,

 

ws

avatar
Explorer

Two more details:  I've put sqljdbc4.jar only on the one node I've logged into with Hue - it's not on the other nodes yet (if that matters?).  Also, I've restarted the Hue and Scoop2 processes via Cloudera Manager.

avatar
Expert Contributor

The sqljdbc4.jar file should be on the node running the Sqoop2 server. It should be put in the directory /var/lib/sqoop2. Then, you just need to restart the Sqoop2 server and it should be available.

 

The UI may not autocomplete it, but it's there.

avatar
Explorer

Bump!

 

I've followed the instructions listed in this thread and I still receive the same error message, "Can't load specified driver". Are there any other ways to troubleshoot this problem?

avatar
Expert Contributor
You can check where Sqoop2 picks up its jars. The file "/etc/sqoop2/tomcat-conf/conf/catalina.properties" will have a "common.loader" property where jars will be picked up. Make sure "/var/lib/sqoop2/*.jar" is in the comma separated list. If its there, then Sqoop2 should be picking up that Jar.