Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How to verify configuration to use Teradata connector specific arguments

avatar
New Contributor

I am getting errors when adding teradata specific connector arguments. What could be the cause ? 

 

Warning: /usr/lib/hcatalog does not exist! HCatalog jobs will fail.

Please set $HCAT_HOME to the root of your HCatalog installation.

15/03/03 21:27:35 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:35 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Error parsing arguments for import:

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --input-method

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: split.by.hash

15/03/03 21:27:35 ERROR tool.BaseSqoopTool: Unrecognized argument: --access-lock

 

sqoop import    --connect jdbc:teradata://tdwc/DATABASE=prod\

                --username xxx --password 'xxx' \

                --table dim_xxx \

                --fields-terminated-by '\001'\

                --target-dir /user/xxx/exports/prod.diml/20150211/dim_sqoop  \

                --split-by option_key \

--compress \

                --num-mappers 5 \

    --input-method split.by.hash\

--access-lock

 

If i am not including the last 2 lines, it works fine and picks up the sqoop teradata connector 

 

15/03/03 21:27:01 INFO sqoop.Sqoop: Running Sqoop version: 1.4.3-cdh4.5.0

15/03/03 21:27:01 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.

15/03/03 21:27:01 INFO teradata.TeradataManagerFactory: Loaded connector factory for 'Cloudera Connector Powered by Teradata' on version 1.3c5

15/03/03 21:27:01 INFO manager.SqlManager: Using default fetchSize of 1000

15/03/03 21:27:01 INFO options.ExtraOptions: Parsing extra arguments

15/03/03 21:27:01 INFO options.OptionsCompatibility: Checking options compatibility

 

I am using 

http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connec...

without cloudera manager. 

 

Thanks in advance,

colin

1 ACCEPTED SOLUTION

avatar
New Contributor

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works. 

 

-- --input-method split.by.hash \

                --access-lock

View solution in original post

2 REPLIES 2

avatar
New Contributor

Found my answer myself. Forgot the -- separator between sqoop settings and teradata specific settings. It now works. 

 

-- --input-method split.by.hash \

                --access-lock

avatar
Mentor
Thanks for sharing the solution!

Indeed, the -- separator lets you pass the connector specific
arguments into the connector itself, than sending it to Sqoop which
would not recognize them.

We cover this need at
http://www.cloudera.com/content/cloudera/en/documentation/connectors/latest/Teradata/Cloudera-Connec...
as "You can control the behavior of the connector by using extra
arguments. Extra arguments must appear at the end of the command,
using -- as the delimiter.".