Created 06-11-2016 01:34 PM
Hi ,
I am doing sqoop import from Teradata to HDFS in hortonworks 2.2 using below command .
Below settings have been done
Placed below the jar files @ /usr/hdp/2.2.0.0-2041/sqoop/lib
1.teradata-connector-1.3.4-hdp2.2.noarch.rpm
2.terajdbc4.jar
3.tdgssconfig.jar
4.hdp-connector-for-teradata-1.4.1.2.3.2.0-2950-distro.tar
Run the below command from /usr/hdp/current/sqoop-client/bin
sqoop import --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --connect jdbc:teradata://192.168.146.130/Database=retail \ --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --table client --username dbc --password dbc \ --target-dir /usr/hue/Abhishek
Getting below error
Warning: /usr/hdp/2.2.0.0-2041/accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/06/11 13:28:04 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.0.0-2041 16/06/11 13:28:04 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/06/11 13:28:04 ERROR sqoop.ConnFactory: Sqoop could not found specified connection manager class org.apache.sqoop.teradata.TeradataConnManager. Please check that you've specified the class correctly. 16/06/11 13:28:04 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:166) at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:249) at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:89) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:589) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) Caused by: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:191) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:146) ... 9 more
------------------------
Please help me on this and let me know the solution.
Regards
Abhishek
Created 06-11-2016 02:08 PM
it seems that teradata connector is not in sqoop classpat ,can you try again by putting teradata-connector-1.*.jar into sqoop classpath and see if it helps
or
if you already unarchived the teradata connector tar into sqoop lib folder then could you please check
jar tvf teradata-connector.jar | grep org.apache.sqoop.teradata.TeradataConnManager
Created 06-11-2016 03:07 PM
Hi,
Are you using HDP provided taradata sqoop connector or Teradata hadoop connector?
Here are jars and sample script if you download the sqoop jars from Teradata Web Site.
please execute below command after providing teradata IP, User ID, Password
sqoop import \
--connect jdbc:teradata://teradataIP/DATABASE=hadoop,CHARSET=UTF8 \
--driver "com.teradata.jdbc.TeraDriver" \
--username abc \
--password ****** \
--query "select * from abc where \$CONDITIONS" \
--null-string '\\N' \ --null-non-string '\\N' \
--hive-delims-replacement '\0D' \
--fields-terminated-by '\001' \
--target-dir /abc/ \
--m 1;
Check the Teradata Sqoop jars at below installed location:
/usr/hdp/2.3.4.0-3485/sqoop/lib/
tdgssconfig.jar
teradata-connector-1.3.2-hadoop210.jar
terajdbc4.jar
sqljdbc4.jar
Note: I downloaded the jars from Teradata Web Site.
try this and let me know.
Diff between HDP sqoop jars & Teradata sqoop jars.
Based on my observation:
HDP Teradata sqoop jars: User ID which you are using in sqoop should have read & write access on Teradata DB, HDP jars will create temporary table in Teradata before moving the data to HDFS
Teradata sqoop jars: Sqoop user ID should have Only read access and write access is not required. Teradata jars wouldn't create any temp tables in Teradata and directly moves the data to HDFS.
Created 06-12-2016 03:56 AM
Thanks @Divakar Annapureddy
I tried the steps what you have suggested .
But again getting the issue
Steps :
1. Place the jar files @ /usr/hdp/2.2.0.0-2041/sqoop/lib
terajdbc4
tdgssconfig
sqljdbc4
teradata-connector-1.3.4-hdp2.2.noarch.rar
2. When i executed the below command
sqoop import \ --connect jdbc:teradata://192.168.146.130/DATABASE=retail,CHARSET=UTF8 \ --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ --driver com.teradata.jdbc.TeraDriver \ --username dbc \ --password dbc \ --table client \ --target-dir /usr/hue/Abhishek/ \ --m 1
getting below error
[root@sandbox bin]# sqoop import \ > --connect jdbc:teradata://192.168.146.130/DATABASE=retail,CHARSET=UTF8 \ > --connection-manager org.apache.sqoop.teradata.TeradataConnManager \ > --driver com.teradata.jdbc.TeraDriver \ > --username dbc \ > --password dbc \ > --table client \ > --target-dir /usr/hue/Abhishek/ \ > --m 1 Warning: /usr/hdp/2.2.0.0-2041/accumulo does not exist! Accumulo imports will fail. Please set $ACCUMULO_HOME to the root of your Accumulo installation. 16/06/12 03:57:43 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5.2.2.0.0-2041 16/06/12 03:57:43 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 16/06/12 03:57:43 ERROR sqoop.ConnFactory: Sqoop could not found specified connection manager class org.apache.sqoop.teradata.TeradataConnManager. Please check that you've specified the class correctly. 16/06/12 03:57:43 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:166) at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:249) at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:89) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:589) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) Caused by: java.lang.ClassNotFoundException: org.apache.sqoop.teradata.TeradataConnManager at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:191) at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:146) ... 9 more
Please help me how to get rid of this
Regards
Abhishek
Created 06-12-2016 04:57 AM
you are getting "ERROR sqoop.ConnFactory: Sqoop could not found specified connection manager class org.apache.sqoop.teradata.TeradataConnManager"
please pass connection manager as well. add below string and give a try
–connection-manager org.apache.sqoop.teradata.TeradataConnManager
Created 06-12-2016 05:41 AM
Please see my 1st question . In that i have mentioned --connection-manager
Created 06-12-2016 04:37 AM
@Abhishek Srivastava could you please post the output of 'jar tvf teradata-connector.*.jar |grep org.apache.sqoop.teradata.TeradataConnManager' after extracting teradata-connector-1.3.4-hdp2.2.noarch.rar
Created 06-12-2016 04:43 AM
Getting below error
[root@sandbox lib]# jar tvf teradata-connector-1.3.4-hdp2.2.noarch.rpm |grep org.apache.sqoop.teradata.TeradataConnManager java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:215) at java.util.zip.ZipFile.<init>(ZipFile.java:145) at java.util.zip.ZipFile.<init>(ZipFile.java:116) at sun.tools.jar.Main.list(Main.java:1004) at sun.tools.jar.Main.run(Main.java:245) at sun.tools.jar.Main.main(Main.java:1177)
Created 06-12-2016 04:51 AM
extract teradata-connector-1.3.4-hdp2.2.noarch.rpm and you will able to see teradata-connector-*.jar, after getting the jar do jar tvf teradata-connector-1.3.4-hdp2.2.noarch.rpm |grep org.apache.sqoop.teradata.TeradataConnManager and post output
Created 06-12-2016 06:46 AM
No output after running this command
[root@sandbox lib]# jar tvf teradata-connector-1.3.4.jar |grep org.apache.sqoop.teradata.TeradataConnManager [root@sandbox lib]#