Sqoop with Oozie error

[#1]

I am running Sqoop action in Oozie, with below script

sqoop import --connect jdbc:sqlserver://SQLM03;database=test --username data_user --password pass1 --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --table table123 -m 1 --target-dir /user/data_user/sqoop

I have placed SQL Server driver jar in Oozie share lib /user/oozie/share/lib/lib_20150429135817/sqoop

I am able to import data using sqoop from command prompt using same script, but getting below error while running from Oozie

 Log Type: stderr
  
  Log Length: 539
  log4j:ERROR Could not find value for key log4j.appender.CLA
  log4j:ERROR Could not instantiate appender named "CLA".
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  No such sqoop tool: sqoop. See 'sqoop help'.
  Intercepting System.exit(1)
  Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

 Tried to search it in previous posts, but no solution found.

Please suggest.

Regards,

Vinod

[#2 · Accepted solution]

No such sqoop tool: sqoop. See 'sqoop help'.

Your Sqoop Action's commands should begin with just the "import" command, and not include "sqoop" as its first argument, i.e. it should look like this:

<command>import --connect …</command>

And not like this, which is how you've specified it:

<command>sqoop import --connect …</command>

The Sqoop Action of Oozie is documented with an example at http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SqoopActionExtension.html

[#3 · Accepted solution]

No such sqoop tool: sqoop. See 'sqoop help'.

Your Sqoop Action's commands should begin with just the "import" command, and not include "sqoop" as its first argument, i.e. it should look like this:

<command>import --connect …</command>

And not like this, which is how you've specified it:

<command>sqoop import --connect …</command>

The Sqoop Action of Oozie is documented with an example at http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SqoopActionExtension.html

[#4]

Thank you Harsh.

It was my bad to overlook the actul commands inside the Sqoop action.

Now I am getting below error,

 Log Type: stderr
 
  Log Length: 3696
  log4j:ERROR Could not find value for key log4j.appender.CLA
  log4j:ERROR Could not instantiate appender named "CLA".
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  Aug 14, 2016 9:47:55 PM com.microsoft.sqlserver.jdbc.SQLServerConnection <init>
  SEVERE: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.
  java.lang.UnsupportedOperationException: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.

I have below drivers in Oozie sahred lib / user/ oozie/ share/ lib/ lib_20150429135817/ sqoop,

sqljdbc.jar, sqljdbc4.jar, sqljdbc41.jar, sqljdb42.jar

Also have below drivers,

Edge node Sqoop2: /var/lib/sqoop2     sqljdbc41.jar

Edge node Sqoop:  /var/lib/sqoop        sqljdbc.jar, sqljdbc4.jar, sqljdbc41.jar, sqljdb42.jar

Sqoop2 Server:      /var/lib/sqoop2       sqljdbc.jar, sqljdbc4.jar, sqljdbc41.jar, sqljdb42.jar

Do I need to add sqljdbc4.jar on Edge node Sqoop2 directory as well remove drivers other than sqljdbc4.jar from all places ? What if in future I want to connect to another SQL Server with different version ?

Are these drivers not come with older version compatibiltiy ?

Thank you,

Vinod

[#5]

You need just one of those jars in the directory, not all of them.


Just use one, the one named sqljdbc41.jar

(The 4.1 JDBC spec is the latest version supported with JDK7).


Remove everything else (especially remove sqljdbc42.jar which won't work unless you upgrade to JDK8) to avoid this problem.

[#6]

I have kept only 1 driver sqljdbc41.jar in all directories and Oozie shared lib path.

But still I am getting below error,

Log Type: stderr
  
  Log Length: 3696
  log4j:ERROR Could not find value for key log4j.appender.CLA
  log4j:ERROR Could not instantiate appender named "CLA".
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  Aug 16, 2016 9:45:42 AM com.microsoft.sqlserver.jdbc.SQLServerConnection <init>
  SEVERE: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.
  java.lang.UnsupportedOperationException: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.

 Even I tried with driver sqljdbc4.jar, but it keep asking other version of driver other than present in directories.

After running a workflow, I realized there is workflow.xml generated in workspace but no job.properties. So I manually created job.properties as below and added in workspace,

 

Below is workflow.xml generated ,

I am not entering any other job properties or parameters in workflow.

Not sure if I am doing something wrong here, as I can not see jobTracker and nameNode actual names is workflow.xml file.

Please guide in this.

[#7 · Accepted solution]

This issue is resolved now.

Took below steps,

1. Removed oozie.libpath line from job.properties

2. Created lib folder in workspace and placed sqljdbc41.jar in lib folder

So total 3 things placed in workspace, job.properties, workflow.xml and lib folder.

[#8]

Hi @Vinod369, 
 I have done same things you mentioned. then I am facing below error. 

java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found

Please help me to resolve this error. 

Regards,

Jay

[#9]

I have this same problem, I too tried follow the method of @Vinod369 and not resolved