question JSON Hive import mapping not working correctly in Archives of Support Questions (Read Only)

JSON Hive import mapping not working correctly

arjan_onwezen — Wed, 08 Nov 2017 00:02:33 GMT

Hi all,

This is my first post here. I have an issue trying to map a JSON field starting with a hash (#) to a Hive column using the (simplified) statement below :

CREATE EXTERNAL TABLE foobar_table (
    foobar STRING
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
WITH SERDEPROPERTIES ( 'mapping.foobar' = '#foobar' ) 
LOCATION '/data/foobar';

Using the following JSON file below:

{"#foobar":"some_value"}

There are no issues when the special character is removed, but this is no solution in the real-world scenario.

Is the fact the mapping isn't working a bug in the code, or am I using the wrong (version of) JsonSerDe. I'm using HDP-2.5.0.0

Hopefully someone can supply with an answer other then renaming the JSON field.

Thanks in advance..

Re: JSON Hive import mapping not working correctly

ccotter — Wed, 08 Nov 2017 19:47:43 GMT

@Arjan Onwezen the hcatalog jsonSerde doesn't look like it uses the SERDEPROPERTIES mappings and only maps exact fieldnames to column names or JSON field names matching Hive internal column names, i.e _col0, _col1. I think the JsonSerde you are referring to is the rcongui Hive-JSON-Serde which should work as you've outlined.

Re: JSON Hive import mapping not working correctly

arjan_onwezen — Wed, 08 Nov 2017 21:54:57 GMT

@Chris Cotter Thank you for our answer. I can confirm the RconGUI's version of JSON Serde works to circumvent the issue.