Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

sqoop.export.records.per.statement parameter doesn't recognize by sqoop

avatar
Rising Star

Hi dear expert!

i'm trying to export data with sqoop.export.records.per.statement parameter.

But for some reasons sqoop doesn't recognize it:

 

sqoop export --direct --connect jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds --table orcl_dpi --export-dir /tmp/dpi  --input-fields-terminated-by ',' --lines-terminated-by '\n' -m 70 --batch -Dsqoop.export.records.per.statement=10000 -Dsqoop.export.statements.per.transaction=100
Warning: /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/22 19:59:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.1
16/01/22 19:59:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Error parsing arguments for export:
16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.records.per.statement=10000
16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.statements.per.transaction=100

 i've tried to remove --direct key (target DB is Oracle), but it also doesn't help:

sqoop export --connect jdbc:oracle:thin:@host:1521:orcl --username user --password pass --table orcl_dpi --export-dir /tmp/dpi  --input-fields-terminated-by ',' --lines-terminated-by '\n' -m 70 --batch -Dsqoop.export.records.per.statement=10000 -Dsqoop.export.statements.per.transaction=100
Warning: /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/22 20:00:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.1
16/01/22 20:00:29 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Error parsing arguments for export:
16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.records.per.statement=10000
16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.statements.per.transaction=100

thank you!

2 ACCEPTED SOLUTIONS

avatar
Mentor
Any -D parameters applying to hadoop or hadoop-related CLI tools, must be
prefixed rather than placed anywhere else. That is, use it in this form
than the one you have (prefixed to the command, vs. suffixed/placed in
middle):

sqoop export -Dsqoop.export.records.per.statement=10000
-Dsqoop.export.statements.per.transaction=100 --direct --connect
jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds
--table orcl_dpi --export-dir /tmp/dpi --input-fields-terminated-by ','
--lines-terminated-by '\n' -m 70 --batch

View solution in original post

avatar
Cloudera Employee
According to the Sqoop documentation, it uses the generic Hadoop parameter passing scheme, which uses a space after the -D parameter denominator. You can check this at the sqoop site here: http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_using_options_files_to_pass_arguments

View solution in original post

2 REPLIES 2

avatar
Mentor
Any -D parameters applying to hadoop or hadoop-related CLI tools, must be
prefixed rather than placed anywhere else. That is, use it in this form
than the one you have (prefixed to the command, vs. suffixed/placed in
middle):

sqoop export -Dsqoop.export.records.per.statement=10000
-Dsqoop.export.statements.per.transaction=100 --direct --connect
jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds
--table orcl_dpi --export-dir /tmp/dpi --input-fields-terminated-by ','
--lines-terminated-by '\n' -m 70 --batch

avatar
Cloudera Employee
According to the Sqoop documentation, it uses the generic Hadoop parameter passing scheme, which uses a space after the -D parameter denominator. You can check this at the sqoop site here: http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_using_options_files_to_pass_arguments