Reply
Explorer
Posts: 14
Registered: ‎12-21-2016

How to escape ^M character in data while doing sqoop import from the Oracle Database

How to escape ^M character in data while doing sqoop import from the Oracle Database

Cloudera Employee
Posts: 32
Registered: ‎08-26-2015

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

You should be able to use the --query option instead of --table so that in the SQL query you can use an Oracle SQL function to remove or convert specific characters.
Explorer
Posts: 14
Registered: ‎12-21-2016

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

If we use the query, the problem is we might be doing the function to replace to only one column and so times these kind of characters can come in other columns as-well. I feel, at runtime with query, these kind of characters can come in from any column.

 

I feel using an argument, will apply to entire table instead of just for a specific column. It would be nice if we can have argument applied to entire table.

 

Thanks

Praveen   

Cloudera Employee
Posts: 32
Registered: ‎08-26-2015

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

There are some escaping arguments in Sqoop[1] but usually they are for
importing to delimited text files, and it would usually be recommended to
import to a structured file format like Parquet or Avro which don't have
these delimiter issues. If you want to escape ^M characters because it
throws off the record delimiter then you might want to look at Parquet or
Avro instead.

[1]
https://sqoop.apache.org/docs/1.4.1-incubating/SqoopUserGuide.html#_large_objects
Highlighted
Explorer
Posts: 14
Registered: ‎12-21-2016

Re: How to escape ^M character in data while doing sqoop import from the Oracle Database

I tried with both the formats (Avro and Parquet), but no luck

Announcements

Currently incubating in Cloudera Labs:

Envelope
HTrace
Ibis
Impyla
Livy
Oryx
Phoenix
Spark Runner for Beam SDK
Time Series for Spark
YCSB