Member since
12-02-2019
19
Posts
4
Kudos Received
2
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 2990 | 04-08-2020 06:13 AM | |
| 2716 | 03-06-2020 08:28 AM |
04-08-2020
06:13 AM
1 Kudo
Hi, after some researches i have find a solution to this issues. The problem was from the Hive table definition for storing data. I was defining some properties of my table like this : hive.createTable("res_idos_0")
.ifNotExists()
.prop("serialization.encoding","UTF-8")
.prop("escape.delim" , "\t")
.column("t_date","TIMESTAMP") But when we are in writeStream and we use special characters, the use of property escape.delim is note supported and we can't save characters correctly. So, i have removed the property escape.delim in my hive table definition and i had also added this line in my code for being certain that file save in HDFS have the right encoding. System.setProperty("file.encoding", "UTF-8")
... View more