Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

sqoop.export.records.per.statement parameter doesn't recognize by sqoop

Solved Go to solution

sqoop.export.records.per.statement parameter doesn't recognize by sqoop

Rising Star

Hi dear expert!

i'm trying to export data with sqoop.export.records.per.statement parameter.

But for some reasons sqoop doesn't recognize it:

 

sqoop export --direct --connect jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds --table orcl_dpi --export-dir /tmp/dpi  --input-fields-terminated-by ',' --lines-terminated-by '\n' -m 70 --batch -Dsqoop.export.records.per.statement=10000 -Dsqoop.export.statements.per.transaction=100
Warning: /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/22 19:59:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.1
16/01/22 19:59:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Error parsing arguments for export:
16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.records.per.statement=10000
16/01/22 19:59:38 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.statements.per.transaction=100

 i've tried to remove --direct key (target DB is Oracle), but it also doesn't help:

sqoop export --connect jdbc:oracle:thin:@host:1521:orcl --username user --password pass --table orcl_dpi --export-dir /tmp/dpi  --input-fields-terminated-by ',' --lines-terminated-by '\n' -m 70 --batch -Dsqoop.export.records.per.statement=10000 -Dsqoop.export.statements.per.transaction=100
Warning: /opt/cloudera/parcels/CDH-5.5.1-1.cdh5.5.1.p1168.923/bin/../lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
16/01/22 20:00:29 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.1
16/01/22 20:00:29 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Error parsing arguments for export:
16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.records.per.statement=10000
16/01/22 20:00:29 ERROR tool.BaseSqoopTool: Unrecognized argument: -Dsqoop.export.statements.per.transaction=100

thank you!

2 ACCEPTED SOLUTIONS

Accepted Solutions

Re: sqoop.export.records.per.statement parameter doesn't recognize by sqoop

Master Guru
Any -D parameters applying to hadoop or hadoop-related CLI tools, must be
prefixed rather than placed anywhere else. That is, use it in this form
than the one you have (prefixed to the command, vs. suffixed/placed in
middle):

sqoop export -Dsqoop.export.records.per.statement=10000
-Dsqoop.export.statements.per.transaction=100 --direct --connect
jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds
--table orcl_dpi --export-dir /tmp/dpi --input-fields-terminated-by ','
--lines-terminated-by '\n' -m 70 --batch

Re: sqoop.export.records.per.statement parameter doesn't recognize by sqoop

Cloudera Employee
According to the Sqoop documentation, it uses the generic Hadoop parameter passing scheme, which uses a space after the -D parameter denominator. You can check this at the sqoop site here: http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_using_options_files_to_pass_arguments
2 REPLIES 2

Re: sqoop.export.records.per.statement parameter doesn't recognize by sqoop

Master Guru
Any -D parameters applying to hadoop or hadoop-related CLI tools, must be
prefixed rather than placed anywhere else. That is, use it in this form
than the one you have (prefixed to the command, vs. suffixed/placed in
middle):

sqoop export -Dsqoop.export.records.per.statement=10000
-Dsqoop.export.statements.per.transaction=100 --direct --connect
jdbc:oracle:thin:@scaj43bda01:1521:orcl --username bds --password bds
--table orcl_dpi --export-dir /tmp/dpi --input-fields-terminated-by ','
--lines-terminated-by '\n' -m 70 --batch

Re: sqoop.export.records.per.statement parameter doesn't recognize by sqoop

Cloudera Employee
According to the Sqoop documentation, it uses the generic Hadoop parameter passing scheme, which uses a space after the -D parameter denominator. You can check this at the sqoop site here: http://sqoop.apache.org/docs/1.4.6/SqoopUserGuide.html#_using_options_files_to_pass_arguments
Don't have an account?
Coming from Hortonworks? Activate your account here