How to verify configuration to use Teradata connector specific arguments

colin2 — Fri, 16 Sep 2022 09:23:09 GMT

I am getting errors when adding teradata specific connector arguments. What could be the cause ?

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

15/03/03 21:27:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Error parsing arguments for import:

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --input-method

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: split.by.hash

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --access-lock

sqoop import --connect jdbc:teradata&colon;//tdwc/DATABASE=prod\

--username xxx --password 'xxx' \

--table dim_xxx \

--fields-terminated-by '\001'\

--target-dir /user/xxx/exports/prod.diml/20150211/dim_sqoop \

--split-by option_key \

--compress \

--num-mappers 5 \

--input-method split.by.hash\

--access-lock

If i am not including the last 2 lines, it works fine and picks up the sqoop teradata connector

15/03/03 21:27:01 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:01 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:01 INFO teradata.TeradataManagerFactory: Loaded connector factory for 'Cloudera Connector Powered by Teradata' on version 1.3c5

15/03/03 21:27:01 INFO manager.SqlManager: Using default fetchSize of 1000

15/03/03 21:27:01 INFO options.ExtraOptions: Parsing extra arguments

15/03/03 21:27:01 INFO options.OptionsCompatibility: Checking options compatibility

I am using

without cloudera manager.

Thanks in advance,

colin

Re: How to verify configuration to use Teradata connector specific arguments

colin2 — Wed, 04 Mar 2015 01:11:16 GMT

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works.

-- --input-method split.by.hash \

--access-lock

Harsh J — Wed, 04 Mar 2015 06:31:34 GMT

Thanks for sharing the solution!

Indeed, the -- separator lets you pass the connector specific
arguments into the connector itself, than sending it to Sqoop which
would not recognize them.

We cover this need at
http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connector-for-Teradata/cctd_use_tpcc.html,
as "You can control the behavior of the connector by using extra
arguments. Extra arguments must appear at the end of the command,
using -- as the delimiter.".