JSON Hive import mapping not working correctly

[#1]

Hi all,

This is my first post here. I have an issue trying to map a JSON field starting with a hash (#) to a Hive column using the (simplified) statement below :

CREATE EXTERNAL TABLE foobar_table (
    foobar STRING
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
WITH SERDEPROPERTIES ( 'mapping.foobar' = '#foobar' ) 
LOCATION '/data/foobar';
 

Using the following JSON file below:

{"#foobar":"some_value"}

There are no issues when the special character is removed, but this is no solution in the real-world scenario.

Is the fact the mapping isn't working a bug in the code, or am I using the wrong (version of) JsonSerDe. I'm using HDP-2.5.0.0

Hopefully someone can supply with an answer other then renaming the JSON field.

Thanks in advance..

[#2 · Accepted solution]

@Arjan Onwezen the hcatalog jsonSerde doesn't look like it uses the SERDEPROPERTIES mappings and only maps exact fieldnames to column names or JSON field names matching Hive internal column names, i.e _col0, _col1. I think the JsonSerde you are referring to is the rcongui Hive-JSON-Serde which should work as you've outlined.

[#3]

@Chris Cotter Thank you for our answer. I can confirm the RconGUI's version of JSON Serde works to circumvent the issue.