Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Sqoop and Oracle Basic Table Compression

Sqoop and Oracle Basic Table Compression

Explorer

Hello- We're using Sqoop 1.4.5 with Oracle 12c and find that Sqoop --direct exports to Oracle are comsuming drastically more space compared to Sqlldr.

 

 

Specifics:

 

Example data set: 24,087,140 records

The destination Oracle table is  partitioned on one column and the table uses basic compression.

 

Results for Sqoop export (command below):  

2,818,572,288 bytes and  86,016 blocks

 

Results for Sqlldr: 

872,415,232 bytes 26,624 blocks

 

Obviously a huge difference for Oracle storage resources.

 

We would appreciate any advice/insight regarding these findings. Please let me know if I can provide any additional information.

 

 

 

Sqoop command:

 

$ sqoop-1.4.5/bin/sqoop export \

-D mapred.child.java.opts='-Xmx4g' \

-D sqoop.export.records.per.statement=5000 \

-D sqoop.export.statements.per.transaction=1000 \

-D mapred.task.timeout=0 \

--connect jdbc:oracle:thin:@test:1521/TEST \

--username test \

--password test \

--direct \

--table TEST \

--input-null-string '\\N' \

--input-null-non-string '\\N' \

--export-dir test_test \

--fields-terminated-by '|' \

-m 18

 

 

Thanks