Reply
New Contributor
Posts: 4
Registered: ‎10-12-2017

Loading csv into Hive adds extra Null

Hello,

 

I am trying to load a csv file to a table in hive

I gave as below to create the table.

 

CREATE TABLE cp
(
ENRL_KEY String
,FMLY_KEY String
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'
WITH SERDEPROPERTIES (
'separatorChar'=',',
'quoteChar'='"',
'escapeChar'='\\'
)
STORED AS TEXTFILE
LOCATION '/data/abc'
TBLPROPERTIES('skip.header.line.count'='1');

 

the output is : each record as a Null record.. so I get exactly double the count.

 

What am I missing here ?  tried with \t,  even line terminator \n etc... no luck.

 

entrl_key

fmly_key

 

NULL

2051012

2374248

 

NULL

29051020

2374248

 

NULL

29051035

2374248

Cloudera Employee
Posts: 213
Registered: ‎03-23-2015

Re: Loading csv into Hive adds extra Null

What version of CDH or Hive are you using?
New Contributor
Posts: 4
Registered: ‎10-12-2017

Re: Loading csv into Hive adds extra Null

CDH 5.10.1 and Hive 0.14.

 

I guess the suspect is encoding.

The csv file I am using is a UTF-16 Little Endian and when I convert into UTF-8 it works fine. But that is not the solution I want as these files cannot be changed it comes from client. the only option is I need to do something during "Create External Table "  I tried all sort of things.

 

Lazyserde

opencsvserde

serialization.encoding = 'WINDOWS-1252' , UTF-16, UTF-16LE etc... nothing works..

 

Has anyone come across this kinda situation, pl let me know.

 

Thanks

A

Announcements