question Re: TDCH export from hive to teradata in Archives of Support Questions (Read Only)

TDCH export from hive to teradata

mraghuram_87 — Thu, 24 Mar 2016 05:45:29 GMT

Hi,

I am using TDCH 1.3 to export data from Hive to Teradata. I added HADOOP_CLASSPATH and HIve home, Hcat home and Sqoop home and HCAT_LIB_JARS and passed this in TDCH command.

export HCAT_LIB_JARS=${HCAT_HOME}/share/hcatalog/hive-hcatalog-core.jar,$HIVE_HOME/lib/hive-cli.jar,$HIVE_HOME/lib/hive-exec.jar,$HIVE_HOME/lib/hive-metastore.jar,$HIVE_HOME/lib/libfb303-0.9.2.jar,$HIVE_HOME/lib/libthrift-0.9.2.jar,$HIVE_HOME/lib/jdo-api-3.0.1.jar

I get the following error. I added all the jars I found the HcatInputFormat class also (the one that is said as missing) in the hive-hcatalog jars ${HCAT_HOME}/share/hcatalog/hive-hcatalog-core.jar. But the path is slightly different in this.

org/apache/hive/hcatalog/mapreduce/HCatInputFormat.class --> highlighted the difference.

But this class is reported as missing when it is in different path. How to fix this?

16/03/23 17:33:41 INFO tool.ConnectorExportTool: java.lang.NoClassDefFoundError: org/apache/hcatalog/mapreduce/HCatInputFormat
        at com.teradata.connector.hcat.processor.HCatInputProcessor.inputPreProcessor(HCatInputProcessor.java:41)

Re: TDCH export from hive to teradata

aervits — Thu, 24 Mar 2016 18:27:08 GMT

Org.apache.hcatalog.mapreduce.HcatInputFormat is pretty old, pre-0.13 Hive. Hcatalog was moved into Hive and vorrwct package is org.apache.hive.hcatalog.mapreduce.HcatInputFormat. you might have version mismatch and or missing libs. Hdp 2.3 is at Hive 1.2.1. Hope that helps, great question.

Here's the HcatLoader wiki, there is info on what jars you need to include rather than guessing yourself.

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Installation

Re: TDCH export from hive to teradata

deepesh1 — Fri, 25 Mar 2016 02:42:24 GMT

Hortonworks Connector for Teradata v1.4.1 is available from downloads, it should work well with HDP-2.3.2.

Re: TDCH export from hive to teradata

mraghuram_87 — Sat, 26 Mar 2016 01:50:54 GMT

Hi I just tried again the next day. I didnt change anything. Same variables for hadoop home hive home, hcat home and Hcat lib jars. Today it worked with both 1.3 and 1.4 TDCH.. Not sure why it didnt work the first time.

Thanks for your time and help!