question Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly. in Archives of Support Questions (Read Only)

How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

lsundar19 — Wed, 07 Sep 2016 18:38:22 GMT

Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

Former Member — Wed, 07 Sep 2016 18:46:44 GMT

Does it help: (for me it did not help, may be i missed something)

https://community.hortonworks.com/questions/54162/why-hive-is-not-able-to-store-special-characters-l.html

Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

lsundar19 — Wed, 07 Sep 2016 20:33:07 GMT

Thanks Jk for your immediate response.

After using "ROW FORMAT SERDE ‘org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe’ WITH SERDEPROPERTIES(“serialization.encoding”=’UTF-8′);" solved the spanish character issue. But we have one more column with values like -10,476.53 because of this column, we had column jumping , this values stored in hive -10 in one column and 476.53 in another column. Do the needful.

Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

bpreachuk — Wed, 07 Sep 2016 21:12:29 GMT

Hi @Sundar Lakshmanan. The input file must use a comma as the field delimiter.

If you can change the format of the input file to use a more non-standard character such as a pipe or a tilde, that would fix the issue. With the lazy simple serde you can specifiy a different field delimiter as another serdeproperty: 'field.delim' = '|'

Otherwise you could use a pig script to pre-process the data to remove comma from that field, or use some hive post-processing to collapse the 2 fields down into one in a new table.

Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

lsundar19 — Wed, 07 Sep 2016 22:07:42 GMT

Thanks bpreachuk,

'field.delim' = '|' also not helping me, But somehow we have fixed the issue with below CSV serde properties :

WITH SERDEPROPERTIES ( "separatorChar" = ",", "quoteChar" = "\"", "escapeChar" = "\\", "serialization.encoding"='ISO-8859-1') LOCATION '/path/' TBLPROPERTIES ( 'store.charset'='ISO-8859-1', 'retrieve.charset'='ISO-8859-1', 'skip.header.line.count'='1');

Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

spanishtoenglis — Wed, 30 Aug 2017 14:54:23 GMT

Change the format of the input file to use a more non-standard character

Steve,

Spanish to English

Re: How can i handle the spanish characters in hive table? Ex : some columns in CSV file have the values like ó, ñ. While load this values to hive,It stored like box �. Please help me on this, i want to create hive table and load this values properly.

happywheelssaz — Thu, 05 Jul 2018 11:02:04 GMT

Well, this is my first visit to your blog! Your blog provided us valuable information .You have done a marvelous job

- happy wheels

I think this is an informative post and it is very useful and knowledgeable. I really enjoyed reading this post. big fan, thank you!

- happy wheels