Support Questions
Find answers, ask questions, and share your expertise

Sqoop Import Error from Teradata

Highlighted

Sqoop Import Error from Teradata

Explorer

Hi ,

I am doing sqoop import from Teradata to HDFS in hortonworks 2.2 using below command .

Below settings have been done

Placed below the jar files @ /usr/hdp/2.2.0.0-2041/sqoop/lib

1.teradata-connector-1.3.4-hdp2.2.noarch.rpm

2.terajdbc4.jar

3.tdgssconfig.jar

4.hdp-connector-for-teradata-1.4.1.2.3.2.0-2950-distro.tar

Run the below command from /usr/hdp/current/sqoop-client/bin

sqoop import --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --connect jdbc:teradata://192.168.146.130/Database=retail \ --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --table client --username dbc --password dbc \ --target-dir /usr/hue/Abhishek

Getting below error

Warning: /usr/hdp/2.2.0.0-2041/accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/06/11 13:28:04 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.0.0-2041 16/06/11 13:28:04 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/06/11 13:28:04 ERROR sqoop.ConnFactory: Sqoop could not found specified connection manager class org.apache.sqoop.teradata.TeradataConnManager. Please check that you've specified the class correctly. 16/06/11 13:28:04 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:166) at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:249) at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:89) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:589) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) Caused by: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:191) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:146) ... 9 more

------------------------

Please help me on this and let me know the solution.

Regards

Abhishek

16 REPLIES 16
Highlighted

Re: Sqoop Import Error from Teradata

@Abhishek Srivastava

it seems that teradata connector is not in sqoop classpat ,can you try again by putting teradata-connector-1.*.jar into sqoop classpath and see if it helps

or

if you already unarchived the teradata connector tar into sqoop lib folder then could you please check

jar tvf teradata-connector.jar | grep org.apache.sqoop.teradata.TeradataConnManager

Highlighted

Re: Sqoop Import Error from Teradata

Hi,

Are you using HDP provided taradata sqoop connector or Teradata hadoop connector?

Here are jars and sample script if you download the sqoop jars from Teradata Web Site.

please execute below command after providing teradata IP, User ID, Password

sqoop import \

--connect jdbc:teradata://teradataIP/DATABASE=hadoop,CHARSET=UTF8 \

--driver "com.teradata.jdbc.TeraDriver" \

--username abc \

--password ****** \

--query "select * from abc where \$CONDITIONS" \

--null-string '\\N' \ --null-non-string '\\N' \

--hive-delims-replacement '\0D' \

--fields-terminated-by '\001' \

--target-dir /abc/ \

--m 1;

Check the Teradata Sqoop jars at below installed location:

/usr/hdp/2.3.4.0-3485/sqoop/lib/

tdgssconfig.jar

teradata-connector-1.3.2-hadoop210.jar

terajdbc4.jar

sqljdbc4.jar

Note: I downloaded the jars from Teradata Web Site.

try this and let me know.

Diff between HDP sqoop jars & Teradata sqoop jars.

Based on my observation:

HDP Teradata sqoop jars: User ID which you are using in sqoop should have read & write access on Teradata DB, HDP jars will create temporary table in Teradata before moving the data to HDFS

Teradata sqoop jars: Sqoop user ID should have Only read access and write access is not required. Teradata jars wouldn't create any temp tables in Teradata and directly moves the data to HDFS.

Highlighted

Re: Sqoop Import Error from Teradata

Explorer

Thanks @Divakar Annapureddy

I tried the steps what you have suggested .

But again getting the issue

Steps :

1. Place the jar files @ /usr/hdp/2.2.0.0-2041/sqoop/lib

terajdbc4

tdgssconfig

sqljdbc4

teradata-connector-1.3.4-hdp2.2.noarch.rar

2. When i executed the below command

sqoop import \ --connect jdbc:teradata://192.168.146.130/DATABASE=retail,CHARSET=UTF8 \ --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --driver com.teradata.jdbc.TeraDriver \ --username dbc \ --password dbc \ --table client \ --target-dir /usr/hue/Abhishek/ \ --m 1

getting below error

[root@sandbox bin]# sqoop import \ > --connect jdbc:teradata://192.168.146.130/DATABASE=retail,CHARSET=UTF8 \ > --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ > --driver com.teradata.jdbc.TeraDriver \ > --username dbc \ > --password dbc \ > --table client \ > --target-dir /usr/hue/Abhishek/ \ > --m 1 Warning: /usr/hdp/2.2.0.0-2041/accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/06/12 03:57:43 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.0.0-2041 16/06/12 03:57:43 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/06/12 03:57:43 ERROR sqoop.ConnFactory: Sqoop could not found specified connection manager class org.apache.sqoop.teradata.TeradataConnManager. Please check that you've specified the class correctly. 16/06/12 03:57:43 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:166) at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:249) at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:89) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:589) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) Caused by: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:191) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:146) ... 9 more

Please help me how to get rid of this

Regards

Abhishek

Highlighted

Re: Sqoop Import Error from Teradata

you are getting "ERROR sqoop.ConnFactory: Sqoop could not found specified connection manager class org.apache.sqoop.teradata.TeradataConnManager"

please pass connection manager as well. add below string and give a try

–connection-manager org.apache.sqoop.teradata.TeradataConnManager

Re: Sqoop Import Error from Teradata

Explorer

Please see my 1st question . In that i have mentioned --connection-manager

Highlighted

Re: Sqoop Import Error from Teradata

@Abhishek Srivastava could you please post the output of 'jar tvf teradata-connector.*.jar |grep org.apache.sqoop.teradata.TeradataConnManager' after extracting teradata-connector-1.3.4-hdp2.2.noarch.rar

Highlighted

Re: Sqoop Import Error from Teradata

Explorer

@Rajkumar Singh

Getting below error

[root@sandbox lib]# jar tvf teradata-connector-1.3.4-hdp2.2.noarch.rpm |grep org.apache.sqoop.teradata.TeradataConnManager java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:215) at java.util.zip.ZipFile.<init>(ZipFile.java:145) at java.util.zip.ZipFile.<init>(ZipFile.java:116) at sun.tools.jar.Main.list(Main.java:1004) at sun.tools.jar.Main.run(Main.java:245) at sun.tools.jar.Main.main(Main.java:1177)

Highlighted

Re: Sqoop Import Error from Teradata

extract teradata-connector-1.3.4-hdp2.2.noarch.rpm and you will able to see teradata-connector-*.jar, after getting the jar do jar tvf teradata-connector-1.3.4-hdp2.2.noarch.rpm |grep org.apache.sqoop.teradata.TeradataConnManager and post output

Highlighted

Re: Sqoop Import Error from Teradata

Explorer

No output after running this command

[root@sandbox lib]# jar tvf teradata-connector-1.3.4.jar |grep org.apache.sqoop.teradata.TeradataConnManager [root@sandbox lib]#