Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

SQOOP issue with teradata connector powered by teradata

avatar
Rising Star

I found the we need to place the jar manually to  /var/lib/sqoop/ 

 

 sqoop-connector-teradata-1.2c5.jar

tdgssconfig.jar

terajdbc4.jar

 

I put theabove jars /var/lib/sqoop/ 

 

is that the correct path?

 

b) As per suggestion i ran the sqoop command in the command prompt

 

 sqoop import --connect jdbc:teradata://absb.222222.COM/DATABASE=ntl_prd_qmtmptbls --username utechntlrpg --password **232 --table par_MOB_APPS --num-mappers 3 --escaped-by '\\' --fields-terminated-by ',' --lines-terminated-by '\n'

 

Warning: /opt/cloudera/parcels/CDH-5.1.0-1.cdh5.1.0.p0.53/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
14/08/21 19:45:57 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4-cdh5.1.0
14/08/21 19:45:57 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/08/21 19:45:57 ERROR tool.BaseSqoopTool: Got error creating database manager: java.io.IOException: No manager for connect string: jdbc:teradata://asas.@@@@.COM/DATABASE=ntl_prd_
    at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:185)
    at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:243)
    at org.apache.sqoop.tool.ImportTool.init(ImportTool.java:84)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:494)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:240)

Can someone pleaselet me know what went wrong??

I  followed the exact steps in the documentation for installation of Teradata connector

 Thanks for your help

 

Thanks,

Amit

1 ACCEPTED SOLUTION

avatar
Rising Star

I fixed the issue...

 

So those who need a solution here are the steps 

 

1) drop the following jars to 

 /var/lib/sqoop/ 

 

 sqoop-connector-teradata-1.2c5.jar

tdgssconfig.jar

terajdbc4.jar

 

2) Go to the /etc/sqoop/conf

 

3) create a drectory

managers.d

 

4) sudo create a file @ /etc/sqoop/conf/managers.d/ cldra_td_connector.txt

 

5) Most important : the content of the file will be

 

com.cloudera.connector.teradata.TeradataManagerFactory = /var/lib/sqoop/sqoop-connector-teradata-1.2c5.jar

And this will do the sqoop installation. 

Sorry the documentation is of no use .....

 

Happy Hadoop-ing

Thanks

Amit

View solution in original post

5 REPLIES 5

avatar
Rising Star

I fixed the issue...

 

So those who need a solution here are the steps 

 

1) drop the following jars to 

 /var/lib/sqoop/ 

 

 sqoop-connector-teradata-1.2c5.jar

tdgssconfig.jar

terajdbc4.jar

 

2) Go to the /etc/sqoop/conf

 

3) create a drectory

managers.d

 

4) sudo create a file @ /etc/sqoop/conf/managers.d/ cldra_td_connector.txt

 

5) Most important : the content of the file will be

 

com.cloudera.connector.teradata.TeradataManagerFactory = /var/lib/sqoop/sqoop-connector-teradata-1.2c5.jar

And this will do the sqoop installation. 

Sorry the documentation is of no use .....

 

Happy Hadoop-ing

Thanks

Amit

avatar
New Contributor

Please help, I m getting following error after follwoing above mentioned steps.

 

16/06/30 17:08:39 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
16/06/30 17:08:39 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/06/30 17:08:39 INFO teradata.TeradataManagerFactory: Loaded connector factory for 'Cloudera Connector Powered by Teradata' on version 1.2c5
16/06/30 17:08:39 INFO manager.SqlManager: Using default fetchSize of 1000
16/06/30 17:08:40 INFO options.ExtraOptions: Parsing extra arguments
16/06/30 17:08:40 INFO options.OptionsCompatibility: Checking options compatibility
16/06/30 17:08:42 INFO tool.CodeGenTool: Beginning code generation
16/06/30 17:08:42 INFO teradata.TeradataManager: Converting table import to query: SELECT * FROM "MT_ACCOUNT_SUB_ACCOUNT"
16/06/30 17:08:44 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/nduggara/hadoop/hadoop-2.6.4
Note: /tmp/sqoop-root/compile/2bbd952f48176e18384fa4bc68346129/MT_ACCOUNT_SUB_ACCOUNT.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
16/06/30 17:08:49 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/2bbd952f48176e18384fa4bc68346129/MT_ACCOUNT_SUB_ACCOUNT.jar
16/06/30 17:08:49 INFO teradata.TeradataManager: Beginning Teradata table based import
16/06/30 17:08:49 INFO mapreduce.ImportJobBase: Beginning import of MT_ACCOUNT_SUB_ACCOUNT
16/06/30 17:08:49 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/nduggara/hadoop/hadoop-2.6.4/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/nduggara/hadoop/hbase/hbase-1.1.5/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/nduggara/hadoop/hive/apache-hive-2.0.1-bin/lib/hive-jdbc-2.0.1-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/nduggara/hadoop/hive/apache-hive-2.0.1-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/06/30 17:08:50 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
16/06/30 17:08:52 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
16/06/30 17:08:52 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
16/06/30 17:08:52 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
16/06/30 17:08:52 INFO mapreduce.TeradataInputProcessor: job setup starts at 1467320932765
16/06/30 17:08:54 INFO mapreduce.TeradataInputProcessor: job setup ends at 1467320934168
16/06/30 17:08:54 INFO mapreduce.TeradataInputProcessor: job setup time is 1s
Exception in thread "main" java.lang.NoClassDefFoundError: org/codehaus/jackson/map/TreeMapper
at com.teradata.hadoop.utils.TeradataSchemaUtils.fieldNamesToJson(TeradataSchemaUtils.java:2046)
at com.teradata.hadoop.db.TeradataConfiguration.setInputSourceFieldNamesArray(TeradataConfiguration.java:846)
at com.teradata.hadoop.utils.TeradataSchemaUtils.setupInputSchema(TeradataSchemaUtils.java:533)
at com.teradata.hadoop.mapreduce.TeradataInputProcessor.setupSchemaMapping(TeradataInputProcessor.java:276)
at com.teradata.hadoop.mapreduce.TeradataSplitByHashInputProcessor.setupSchemaMapping(TeradataSplitByHashInputProcessor.java:85)
at com.teradata.hadoop.mapreduce.TeradataInputProcessor.setup(TeradataInputProcessor.java:56)
at com.teradata.hadoop.mapreduce.TeradataSplitByHashInputProcessor.setup(TeradataSplitByHashInputProcessor.java:51)
at com.teradata.hadoop.job.TeradataHdfsFileImportJob.runJob(TeradataHdfsFileImportJob.java:207)
at com.teradata.hadoop.tool.TeradataJobRunner.runImportJob(TeradataJobRunner.java:119)
at com.cloudera.connector.teradata.imports.BaseImportJob.doSubmitJob(BaseImportJob.java:43)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)
at com.cloudera.connector.teradata.TeradataManager.importTable(TeradataManager.java:273)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.map.TreeMapper
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 21 more

 

I am getting this error

avatar
New Contributor

Amit,

 

Where can you find these 2 jars. 

tdgssconfig.jar

terajdbc4.jar

 

Teradata site is asking credentials to download

 

avatar
Champion

Sqoop - Teradata connector

pre requist  - u need a cloudera login u can create one for free. 

 

 https://www.cloudera.com/downloads/connectors/sqoop/teradata/1-6c5.html

 

 

 

 

U need Teradata login credentials . just register with them . 

 

https://www.teradata.com/registration.aspx

https://downloads.teradata.com/download/connectivity/jdbc-driver

 

https://downloads.teradata.com/download/connectivity/jdbc-driver

avatar
New Contributor

Hi @AmitD ,

I did the same steps that worked for you. But I am getting the below error. Any idea what can be the reason ?

 

19/10/28 13:58:16 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.11.1
19/10/28 13:58:16 INFO teradata.TeradataManagerFactory: Loaded connector factory for 'Cloudera Connector Powered by Teradata' on version 1.7c6
19/10/28 13:58:16 ERROR tool.BaseSqoopTool: Got error creating database manager: java.lang.ClassCastException: com.cloudera.connector.teradata.TeradataManagerFactory cannot be cast to com.cloudera.sqoop.manager.ManagerFactory
at org.apache.sqoop.ConnFactory.instantiateFactories(ConnFactory.java:98)
at org.apache.sqoop.ConnFactory.<init>(ConnFactory.java:63)
at com.cloudera.sqoop.ConnFactory.<init>(ConnFactory.java:36)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:270)
at org.apache.sqoop.tool.EvalSqlTool.run(EvalSqlTool.java:56)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)