Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

sqoop import when source table contains string with CRLF

avatar
Explorer
Hi all,

I have to import a mysql table into hive environment.

The table has a "notes" column with free text, including the possibility for the user to insert CRLF.

 
when I try to sqoop import that table, it seems the sqoop command interprets this CRLF as a end of row....

 
Then everything, from that point on, is not correctly imported: I mean that the CRLF is the end of the row and the first column of the next row in filled in with the "rest" of free text and so on

 
I use this command:
sqoop import --connect jdbc:mysql://myserver:3306/mydb --username myusername --password mypwd
--table mytable --hive-import --hive-table mytable --hive-database mydb --as-textfile
--warehouse-dir /user/mydb --fields-terminated-by '|' --direct -m 1 --driver com.mysql.jdbc.Driver


1 ACCEPTED SOLUTION

avatar
Explorer
Hi all, finally found a solution actually it was under my eyes but didn't get soon# I simply added to my sqoop import command the option: --hive-drop-import-delims bye!

View solution in original post

1 REPLY 1

avatar
Explorer
Hi all, finally found a solution actually it was under my eyes but didn't get soon# I simply added to my sqoop import command the option: --hive-drop-import-delims bye!