Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

How to verify configuration to use Teradata connector specific arguments

avatar
Frequent Visitor

I am getting errors when adding teradata specific connector arguments. What could be the cause ? 

 

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

15/03/03 21:27:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Error parsing arguments for import:

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --input-method

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: split.by.hash

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --access-lock

 

sqoop import    --connect jdbc:teradata://tdwc/DATABASE=prod\

                --username xxx --password 'xxx' \

                --table dim_xxx \

                --fields-terminated-by '\001'\

                --target-dir /user/xxx/exports/prod.diml/20150211/dim_sqoop  \

                --split-by option_key \

--compress \

                --num-mappers 5 \

    --input-method split.by.hash\

--access-lock

 

If i am not including the last 2 lines, it works fine and picks up the sqoop teradata connector 

 

15/03/03 21:27:01 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:01 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:01 INFO teradata.TeradataManagerFactory: Loaded connector factory for 'Cloudera Connector Powered by Teradata' on version 1.3c5

15/03/03 21:27:01 INFO manager.SqlManager: Using default fetchSize of 1000

15/03/03 21:27:01 INFO options.ExtraOptions: Parsing extra arguments

15/03/03 21:27:01 INFO options.OptionsCompatibility: Checking options compatibility

 

I am using 

http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connec...

without cloudera manager. 

 

Thanks in advance,

colin

1 ACCEPTED SOLUTION

avatar
Frequent Visitor

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works. 

 

-- --input-method split.by.hash \

                --access-lock

View solution in original post

2 REPLIES 2

avatar
Frequent Visitor

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works. 

 

-- --input-method split.by.hash \

                --access-lock

avatar
Mentor
Thanks for sharing the solution!

Indeed, the -- separator lets you pass the connector specific
arguments into the connector itself, than sending it to Sqoop which
would not recognize them.

We cover this need at
http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connec...
as "You can control the behavior of the connector by using extra
arguments. Extra arguments must appear at the end of the command,
using -- as the delimiter.".