Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop with Oozie error

SOLVED Go to solution

Sqoop with Oozie error

Contributor

I am running Sqoop action in Oozie, with below script

 

sqoop import --connect jdbc:sqlserver://SQLM03;database=test --username data_user --password pass1 --driver com.microsoft.sqlserver.jdbc.SQLServerDriver --table table123 -m 1 --target-dir /user/data_user/sqoop

 

I have placed SQL Server driver jar in Oozie share lib /user/oozie/share/lib/lib_20150429135817/sqoop

I am able to import data using sqoop from command prompt using same script, but getting below error while running from Oozie

 Log Type: stderr
  
  Log Length: 539
  log4j:ERROR Could not find value for key log4j.appender.CLA
  log4j:ERROR Could not instantiate appender named "CLA".
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  No such sqoop tool: sqoop. See 'sqoop help'.
  Intercepting System.exit(1)
  Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

 Tried to search it in previous posts, but no solution found.

 

Please suggest.

 

Regards,

Vinod

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: Sqoop with Oozie error

Master Guru
No such sqoop tool: sqoop. See 'sqoop help'.

Your Sqoop Action's commands should begin with just the "import" command, and not include "sqoop" as its first argument, i.e. it should look like this:

<command>import --connect …</command>

And not like this, which is how you've specified it:

<command>sqoop import --connect …</command>

The Sqoop Action of Oozie is documented with an example at http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SqoopActionExtension.html

Re: Sqoop with Oozie error

Contributor

This issue is resolved now.

Took below steps,

1. Removed oozie.libpath line from job.properties

2. Created lib folder in workspace and placed sqljdbc41.jar in lib folder

So total 3 things placed in workspace, job.properties, workflow.xml and lib folder.

7 REPLIES 7

Re: Sqoop with Oozie error

Master Guru
No such sqoop tool: sqoop. See 'sqoop help'.

Your Sqoop Action's commands should begin with just the "import" command, and not include "sqoop" as its first argument, i.e. it should look like this:

<command>import --connect …</command>

And not like this, which is how you've specified it:

<command>sqoop import --connect …</command>

The Sqoop Action of Oozie is documented with an example at http://archive.cloudera.com/cdh5/cdh/5/oozie/DG_SqoopActionExtension.html

Re: Sqoop with Oozie error

Contributor

Thank you Harsh.

It was my bad to overlook the actul commands inside the Sqoop action.

 

Now I am getting below error,

 

 Log Type: stderr
 
  Log Length: 3696
  log4j:ERROR Could not find value for key log4j.appender.CLA
  log4j:ERROR Could not instantiate appender named "CLA".
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  Aug 14, 2016 9:47:55 PM com.microsoft.sqlserver.jdbc.SQLServerConnection <init>
  SEVERE: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.
  java.lang.UnsupportedOperationException: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.

 

 

I have below drivers in Oozie sahred lib / user/ oozie/ share/ lib/ lib_20150429135817/ sqoop,

sqljdbc.jar, sqljdbc4.jar, sqljdbc41.jar, sqljdb42.jar

 

Also have below drivers,

Edge node Sqoop2: /var/lib/sqoop2     sqljdbc41.jar

Edge node Sqoop:  /var/lib/sqoop        sqljdbc.jar, sqljdbc4.jar, sqljdbc41.jar, sqljdb42.jar

Sqoop2 Server:      /var/lib/sqoop2       sqljdbc.jar, sqljdbc4.jar, sqljdbc41.jar, sqljdb42.jar

 

Do I need to add sqljdbc4.jar on Edge node Sqoop2 directory as well remove drivers other than sqljdbc4.jar from all places ? What if in future I want to connect to another SQL Server with different version ?

Are these drivers not come with older version compatibiltiy ?

 

Thank you,

Vinod

Re: Sqoop with Oozie error

Master Guru

You need just one of those jars in the directory, not all of them.


Just use one, the one named sqljdbc41.jar

(The 4.1 JDBC spec is the latest version supported with JDK7).


Remove everything else (especially remove sqljdbc42.jar which won't work unless you upgrade to JDK8) to avoid this problem.

Re: Sqoop with Oozie error

Contributor

I have kept only 1 driver sqljdbc41.jar in all directories and Oozie shared lib path.

But still I am getting below error,

Log Type: stderr
  
  Log Length: 3696
  log4j:ERROR Could not find value for key log4j.appender.CLA
  log4j:ERROR Could not instantiate appender named "CLA".
  log4j:WARN No appenders could be found for logger (org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider).
  log4j:WARN Please initialize the log4j system properly.
  log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
  Aug 16, 2016 9:45:42 AM com.microsoft.sqlserver.jdbc.SQLServerConnection <init>
  SEVERE: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.
  java.lang.UnsupportedOperationException: Java Runtime Environment (JRE) version 1.7 is not supported by this driver. Use the sqljdbc4.jar class library, which provides support for JDBC 4.0.

 Even I tried with driver sqljdbc4.jar, but it keep asking other version of driver other than present in directories.

 

After running a workflow, I realized there is workflow.xml generated in workspace but no job.properties. So I manually created job.properties as below and added in workspace,

 

 job.properties_p.jpg

Below is workflow.xml generated ,

workflow.xml_p.jpg

 

I am not entering any other job properties or parameters in workflow.

Not sure if I am doing something wrong here, as I can not see jobTracker and nameNode actual names is workflow.xml file.

 

Please guide in this.

 

 

 

 

 

Re: Sqoop with Oozie error

Contributor

This issue is resolved now.

Took below steps,

1. Removed oozie.libpath line from job.properties

2. Created lib folder in workspace and placed sqljdbc41.jar in lib folder

So total 3 things placed in workspace, job.properties, workflow.xml and lib folder.

Re: Sqoop with Oozie error

Contributor

Hi @Vinod369
 I have done same things you mentioned. then I am facing below error. 

java.lang.ClassNotFoundException: Class org.apache.oozie.action.hadoop.SqoopMain not found

 

Please help me to resolve this error. 

 

Regards,

Jay

Highlighted

Re: Sqoop with Oozie error

New Contributor
I have this same problem, I too tried follow the method of @Vinod369 and not resolved