Support Questions
Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Innovation Accelerator group hub.

Problems importing tables from SAP that have / characters in the table name

Contributor

Is there a way to make sqoop accept / characters in table and field names?

I have tried the following:

leaving the name as-is: --table /BI0/TCUSTOMER

wrapping in quotes: --table "/BI0/TCUSTOMER"

escaping each /: --table "\/BI0\/TCUSTOMER"

It produces this error each time for all three options above:

com.sap.db.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: [257]: sql syntax error: incorrect syntax near "\": line 1 col 17 (at pos 17)

16 REPLIES 16

Have you tried using --query using a specific select statement to pull data?

Contributor

So I found that if i force a query using --query it will work, but if I let sqoop generate the query it does not.

Example --query "select * from \"BI0/TCUSTOMER\"" will work, but referring to the table in the same way by using the --table \"BI0/TCUSTOMER\" argument does not work. So the problem must be when sqoop generates the SQL query on its own, it is not syntactically correct.

If there was a way to see the query it was generating that would be very helpful to troubleshoot it, but as far as I know there is not.

Contributor

Yes, doing that gave the same error.

@Josh Persinger

Please use like this, it will resolve the issue.

--table "TestSchema\".\"/BIC/AZDSOV1_100"

Similar to bug https://issues.apache.org/jira/browse/SQOOP-1722

I still believe this can be fix through different syntax since I don't have SAP installed with SAP driver so can't test it with multiple changes locally.

Please give another try with below syntax.

"SCHEMA.\"/BI0/TCUSTOMER\""

Did you tried the last suggested syntax or are you able to resolve this issue?

Contributor

Yes I tried the above. That somewhat fixed my problem, it seems to accept the syntax now, but I believe it is now literally interpreting the quotes as part of the table name, so when it goes to SAP to find that table, it can't find it because the quotes aren't actually there in the SAP table name...

16/06/08 15:10:31 ERROR tool.ImportTool: Encountered IOException running import job: java.io.FileNotFoundException: /tmp/sqoop-hdfs/compile/d26f755016cd4c734711de4a2550ca46/SCHEMA."/BI0/TCUSTOMER".jar (No such file or directory)

Ok then we may need to remove the initial double quote.

SCHEMA.\"/BI0/TCUSTOMER\"

Contributor

Got the same error:

16/06/08 16:20:49 ERROR tool.ImportTool: Encountered IOException running import job: java.io.FileNotFoundException: /tmp/sqoop-hdfs/compile/2fb644a9751cabd5c187540154e2e696/SAPR3."/BI0/TCUSTOMER".jar (No such file or directory)

So it seems that SAP JDBC error is gone but somehow sqoop is not resolving the double quotes. Will do search locally and get back to you.

Contributor

Any updates on this?

Sorry didn't get cycle to work on this. I will try it in couple of days. Also lets wait if other experts comment on this.

Contributor

That still doesn't work.

Here's my script:

sqoop --options-file options.txt \

--table "SCHEMA\".\"/BI0/TCUSTOMER"

--hive-import

--hive-overwrite

--split-by CUSTOMER

--hive-database DATABASE

Produces this error:

com.sap.db.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: [257]: sql syntax error: incorrect syntax near "/": line 1 col 26 (at pos 26)

Explorer

Can you try it with backqoutes? Something like this - \`tablename\`

Contributor

I tried that and got this error:

ERROR manager.SqlManager: Error executing statement: com.sap.db.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: [257]: sql syntax error: incorrect syntax near "AS": line 1 col 19 (at pos 19) com.sap.db.jdbc.exceptions.JDBCDriverException: SAP DBTech JDBC: [257]: sql syntax error: incorrect syntax near "AS": line 1 col 19 (at pos 19)

Contributor

I can confirm what @Josh Persinger is saying. The only way to get tables with forward slashes ('/') (and actually colons too) in the tablename from SAP into hadoop hdfs/hive is by using the --query statement.

Some other things I found out when importing from SAP HANA:

  1. a table name can be something like 'MSG\TABLENAME' or even worse: '[SCHEMA]::database.[TABLENAME]'. Just make sure you put the complete tablename between escaped double quotes:

    eg: \"/SOMETING/TABLENAME\" or \"[SCHEMA]::database.[TABLENAME]\"

  2. we needed to add there where clause '\$CONDITIONS' even though we did a select * without any filters.
  3. when limiting the result with a where clause the values have to be between single quotes:

    eg. WHERE DDLANGUAGE='E'

  4. SAP columns can contain empty values called SPACE (not the same as NULL) (shown as a '?' in the webIDE). If you want to exclude them use the where clause <>'' (just two singlequotes following each other):

    WHERE DDLANGUAGE<>''

  5. When making your command more readible I had to keep one extra parameter after the --query parameter. When I moved the --hive-import to the next line the command would fail (I think due to the ending quotes of the query.

The result should look something like this:

sqoop import --connect "jdbc:sap://[SERVER]:30015/?currentschema=[SCHEMA]" \

--username [USERNAME] \

--password-file file:///[PATH]/.pwd_file \

--driver com.sap.db.jdbc.Driver \

--query "select * from \"[/TABLENAME/WITH/FORWARDSLASHES]\" WHERE DDLANGUAGE='E' and [COLUMNNAME]<>'' and \$CONDITIONS" --hive-import \

--hive-database [HIVEDATABASE] \

--hive-table [TABLENAME] \

--hive-delims-replacement ' ' \

--fields-terminated-by '|' \

--lines-terminated-by '\n' \

--hive-overwrite --num-mappers 1