Created 05-06-2017 06:00 PM
I have imported a table using Sqoop from oracle data base which contains the 80000 records. After import done I shocked total around 85000 records are available in the outputdir -m-part000 .
I used the textfile as format with delimited "," and newline "\n" , I look into the output file ..I found that a single record data splits and comes as 2nd row. but in the source table first and second rows data is a single record.
I thought some newline or delimited issues occurred using sqoop..please advice how to handle..
@Mahesh K get the output of DBMS_METADATA.GET_DDL('TABLE','EMP','SCOTT') for the source table in Oracle, you'd have to compare both the layouts first, for delimiter, nulls, quotes if any. Text is certainly a good choice in hive, however, the issue looks like field delimiter or linefeed.
Can you share one or two rows of the records you believe were split as two records ?
Created 05-08-2017 06:16 AM
Created 05-08-2017 06:15 AM
Hi Shyam ,
Thanks for the repsonse...
I have looked into the DB table one of the single column having text as below ..
"Not directly related to this loss event, but controls have been in place as below
- Have set up hard limit to prevent fat finger order error
- Have double checked the message poped up or alarmed from the system"
Hence , The same data in part-m-00000 coming as below 3 records..
1- Not directly related to this loss event, but controls have been in place as below
2 - Have set up hard limit to prevent fat finger order error
3- Have double checked the message poped up or alarmed from the system
at runtime how we can avoid..since i am using - --import-all-tables using sqoop.