Cloudera Labs
Provide feedback on Cloudera Labs
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to escape ^M character in data while doing sqoop import from the Oracle Database

Highlighted

How to escape ^M character in data while doing sqoop import from the Oracle Database

Rising Star

How to escape ^M character in data while doing sqoop import from the Oracle Database

4 REPLIES 4

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

Rising Star
You should be able to use the --query option instead of --table so that in the SQL query you can use an Oracle SQL function to remove or convert specific characters.

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

Rising Star

If we use the query, the problem is we might be doing the function to replace to only one column and so times these kind of characters can come in other columns as-well. I feel, at runtime with query, these kind of characters can come in from any column.

 

I feel using an argument, will apply to entire table instead of just for a specific column. It would be nice if we can have argument applied to entire table.

 

Thanks

Praveen   

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

Rising Star
There are some escaping arguments in Sqoop[1] but usually they are for
importing to delimited text files, and it would usually be recommended to
import to a structured file format like Parquet or Avro which don't have
these delimiter issues. If you want to escape ^M characters because it
throws off the record delimiter then you might want to look at Parquet or
Avro instead.

[1]
https://sqoop.apache.org/docs/1.4.1-incubating/SqoopUserGuide.html#_large_objects

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

Rising Star

I tried with both the formats (Avro and Parquet), but no luck