question Sqoop : Lzo compression not working for importing data from oracle to HDFS in Archives of Support Questions (Read Only)

Sqoop : Lzo compression not working for importing data from oracle to HDFS

arkaprova — Wed, 14 Sep 2016 14:10:51 GMT

Sqoop Command :

sqoop import --connect jdbc:oracle:thin:@//**.***.***.***:1521/*** --username ***** --password ******* --table COUNTRIES --target-dir /user/aps/test --compress --compression-codec com.hadoop.compression.lzo.LzopCodec -m 1

I am getting below error

16/09/14 12:32:41 ERROR tool.ImportTool: Encountered IOException running import job: com.cloudera.sqoop.io.UnsupportedCodecException: com.hadoop.compression.lzo.LzopCodec at org.apache.sqoop.io.CodecMap.getCodecClassName(CodecMap.java:79) at org.apache.sqoop.io.CodecMap.getCodec(CodecMap.java:101) at com.cloudera.sqoop.io.CodecMap.getCodec(CodecMap.java:64) at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:116) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673) at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:445) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) at org.apache.sqoop.Sqoop.run(Sqoop.java:148) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235) at org.apache.sqoop.Sqoop.main(Sqoop.java:244)

Please help.

Thanks, Arkaprova

Re: Sqoop : Lzo compression not working for importing data from oracle to HDFS

pvillard — Wed, 14 Sep 2016 15:30:46 GMT

Hi,

Could you try:

sqoop import --connect jdbc:oracle:thin:@//**.***.***.***:1521/*** --username ***** --password ******* --table COUNTRIES --target-dir /user/aps/test --compress --compression-codec LZOP -m 1

Or LZO if you want LZO instead of LZOP.

Re: Sqoop : Lzo compression not working for importing data from oracle to HDFS

arkaprova — Wed, 14 Sep 2016 16:09:34 GMT

@Pierre Villard

I am getting below error now if I use --compression-codec LZOP

16/09/14 14:36:29 ERROR tool.ImportTool: Encountered IOException running import job: com.cloudera.sqoop.io.UnsupportedCodecException: LZOP at org.apache.sqoop.io.CodecMap.getCodecClassName(CodecMap.java:79) at org.apache.sqoop.io.CodecMap.getCodec(CodecMap.java:101) at com.cloudera.sqoop.io.CodecMap.getCodec(CodecMap.java:64) at org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:116) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:259) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673) at org.apache.sqoop.manager.OracleManager.importTable(OracleManager.java:445) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) at org.apache.sqoop.Sqoop.run(Sqoop.java:148) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235) at org.apache.sqoop.Sqoop.main(Sqoop.java:244)

Please suggest.

Re: Sqoop : Lzo compression not working for importing data from oracle to HDFS

pvillard — Wed, 14 Sep 2016 16:27:59 GMT

Could you try with lower case?

--compression-codec lzop

Based on the code:

https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/io/CodecMap.java

it may be case sensitive...

Re: Sqoop : Lzo compression not working for importing data from oracle to HDFS

arkaprova — Wed, 14 Sep 2016 17:41:07 GMT

@Pierre Villard

Thanks a lot. This is working now with lower case.

Re: Sqoop : Lzo compression not working for importing data from oracle to HDFS

arkaprova — Wed, 14 Sep 2016 17:43:29 GMT

Thanks a lot . This is working for lower case.