Member since
09-30-2021
2
Posts
0
Kudos Received
0
Solutions
09-30-2021
10:39 AM
Thanks for your response Ashish, If you see my test data , its having two double quotes (extra double quote ) in column values . thats where its creating problems. "1","peter","He is Data enginer"", "Senior Engineer" "2","Anee","Hadoop Engineer"","Lead" "3","James","Data, Architect"","Sr Architect"
... View more
09-30-2021
07:33 AM
I need to load the CSV data into hive table but i am facing issues with embedded double quotes in few column values as well embedded commas in other columns . Because of this, wherever embedded double quotes and embedded commas are occured , the data from there not loading properly and filled with nulls. I have tried using below openCSV serde options and all. But no luck! ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde' WITH SERDEPROPERTIES ( "separatorChar" = "\t", "quoteChar" = "'", "escapeChar" = "\\" ) theCSV data looks like this id,name,description,role "1","peter","He is Data enginer"", "Senior Engineer" "2","Anee","Hadoop Engineer"","Lead" "3","James","Data, Architect"","Sr Architect" hive data should look like this after loading Id Name Description Role 1 peter He is Data Engineer Senior Engineer 2 anee Hadoop Engineer Lead 3 james Data, Architect Sr Architect
... View more
Labels:
- Labels:
-
Apache Hive