Support Questions

Find answers, ask questions, and share your expertise

How to verify configuration to use Teradata connector specific arguments

avatar
New Contributor

I am getting errors when adding teradata specific connector arguments. What could be the cause ? 

 

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

15/03/03 21:27:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Error parsing arguments for import:

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --input-method

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: split.by.hash

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --access-lock

 

sqoop import    --connect jdbc:teradata://tdwc/DATABASE=prod\

                --username xxx --password 'xxx' \

                --table dim_xxx \

                --fields-terminated-by '\001'\

                --target-dir /user/xxx/exports/prod.diml/20150211/dim_sqoop  \

                --split-by option_key \

--compress \

                --num-mappers 5 \

    --input-method split.by.hash\

--access-lock

 

If i am not including the last 2 lines, it works fine and picks up the sqoop teradata connector 

 

15/03/03 21:27:01 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:01 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:01 INFO teradata.TeradataManagerFactory: Loaded connector factory for 'Cloudera Connector Powered by Teradata' on version 1.3c5

15/03/03 21:27:01 INFO manager.SqlManager: Using default fetchSize of 1000

15/03/03 21:27:01 INFO options.ExtraOptions: Parsing extra arguments

15/03/03 21:27:01 INFO options.OptionsCompatibility: Checking options compatibility

 

I am using 

http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connec...

without cloudera manager. 

 

Thanks in advance,

colin

1 ACCEPTED SOLUTION

avatar
New Contributor

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works. 

 

-- --input-method split.by.hash \

                --access-lock

View solution in original post

2 REPLIES 2

avatar
New Contributor

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works. 

 

-- --input-method split.by.hash \

                --access-lock

avatar
Mentor
Thanks for sharing the solution!

Indeed, the -- separator lets you pass the connector specific
arguments into the connector itself, than sending it to Sqoop which
would not recognize them.

We cover this need at
http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connec...
as "You can control the behavior of the connector by using extra
arguments. Extra arguments must appear at the end of the command,
using -- as the delimiter.".