Support Questions
Find answers, ask questions, and share your expertise

Sqoop Java 1.4.6 Import fails while writing to Parquet file in HDFS. But works with Text file

New Contributor

I am trying to import data from Oracle to HDFS using sqoop java 1.4.6

My hadoop version 2.6.0-cdh 5.14.4

Sqoop version 1.4.6-cdh 5.14.4

 

Dependencies i have used in pom:

sqoop 1.4.6-cdh 5.14.4

ojdbc8 12.2.0.1

hadoop-mapreduce-client-jobclient 2.6.0-cdh 5.14.4

hadoop-mapreduce-client-common 2.6.0-cdh 5.14.4

kite-data-mapreduce 1.0.0-cdh 5.14.4

kite-data-code 1.0.0-cdh 5.14.4

kite-hadoop-compatibility 1.0.0-cdh 5.14.4

httpclient 4.5.9

commons-lang3 3.9

 

The Error I am getting while writing as parquet file:

org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: class org.kitesdk.data.mapreduce.DatasetKeyOutputFormat not found

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:531)

 

Please note that I am able to write it to output when I pass argument as textfile (--textfile). 

The Error is only encountered when i try to write it as parquetfile (--parquetfile).

 

Kindly let me know how to correct this. 

0 REPLIES 0