how to read csv have comma in cell and new line character as well . like i have somme columns with description (hi,how are you).
i am trying to read with spark in scala .
I would recommend not using CSV in your case. If you have commas in the fields then you can't really delimit them with commas because, as you have noticed, you will have field breaks in the middle of a field.
Can you get the source data exported some other way?
Where is the data coming from? You could use a binary format like Avro or Parquet if your source system can export that way. If you MUST have a text file with a delimiter, you need a delimiter that is not anywhere in the data.
Have you tried using openSerde In DDL?
Create Table tablename(columns)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde';
actually, i am getting file from ftp and reading using spark. i didn't try to push directly to hive because some enrichment i have to made. this is a big problem for me.