Support Questions
Find answers, ask questions, and share your expertise

Sqoop Java 1.4.6 Import fails while writing to Parquet file in HDFS. But works with Text file

Sqoop Java 1.4.6 Import fails while writing to Parquet file in HDFS. But works with Text file

New Contributor

I am trying to import data from Oracle to HDFS using sqoop java 1.4.6

My hadoop version 2.6.0-cdh 5.14.4

Sqoop version 1.4.6-cdh 5.14.4

 

Dependencies i have used in pom:

sqoop 1.4.6-cdh 5.14.4

ojdbc8 12.2.0.1

hadoop-mapreduce-client-jobclient 2.6.0-cdh 5.14.4

hadoop-mapreduce-client-common 2.6.0-cdh 5.14.4

kite-data-mapreduce 1.0.0-cdh 5.14.4

kite-data-code 1.0.0-cdh 5.14.4

kite-hadoop-compatibility 1.0.0-cdh 5.14.4

httpclient 4.5.9

commons-lang3 3.9

 

The Error I am getting while writing as parquet file:

org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: class org.kitesdk.data.mapreduce.DatasetKeyOutputFormat not found

at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$2.call(MRAppMaster.java:531)

 

Please note that I am able to write it to output when I pass argument as textfile (--textfile). 

The Error is only encountered when i try to write it as parquetfile (--parquetfile).

 

Kindly let me know how to correct this.