Skip to main content
marcodiv
Gold ⭐️
February 1, 2019
Solved

sqoop import when source table contains string with CRLF

Hi all,

I have to import a mysql table into hive environment.

The table has a "notes" column with free text, including the possibility for the user to insert CRLF.

 
when I try to sqoop import that table, it seems the sqoop command interprets this CRLF as a end of row....

 
Then everything, from that point on, is not correctly imported: I mean that the CRLF is the end of the row and the first column of the next row in filled in with the "rest" of free text and so on

 
I use this command:

sqoop import --connect jdbc:mysql://myserver:3306/mydb --username myusername --password mypwd
--table mytable --hive-import --hive-table mytable --hive-database mydb --as-textfile
--warehouse-dir /user/mydb --fields-terminated-by '|' --direct -m 1 --driver com.mysql.jdbc.Driver



marcodiv
marcodivAnswer
Gold ⭐️
February 1, 2019
Hi all, finally found a solution actually it was under my eyes but didn't get soon# I simply added to my sqoop import command the option: --hive-drop-import-delims bye!

1 reply

marcodiv
marcodivAuthorAnswer
Gold ⭐️
February 1, 2019
Hi all, finally found a solution actually it was under my eyes but didn't get soon# I simply added to my sqoop import command the option: --hive-drop-import-delims bye!
Translate