There is a design pattern being developed at our organization wherein
one of the scenarios is to load data from an Oracle database on to
Hive. The Hive table is defined as ORC format with ZLIB compression.
However, there is an error occurring when the Oracle source contains
data with newline '\n' characters within text columns. Also, when we
attempt to change 'LINES TERMINTED BY' to anything other than '\n' it is
Could you please elaborate on your statement - "migrate away from
character separated files to a modern format like ORC or avro" ?
Any assistance or pseudocode will be much appreciated.
I could provide details on the configuration in use, if required.