Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

JSON Hive import mapping not working correctly

avatar
New Contributor

Hi all,

This is my first post here. I have an issue trying to map a JSON field starting with a hash (#) to a Hive column using the (simplified) statement below :

CREATE EXTERNAL TABLE foobar_table (
    foobar STRING
)
ROW FORMAT SERDE 'org.apache.hive.hcatalog.data.JsonSerDe'
WITH SERDEPROPERTIES ( 'mapping.foobar' = '#foobar' ) 
LOCATION '/data/foobar';
 

Using the following JSON file below:

{"#foobar":"some_value"}

There are no issues when the special character is removed, but this is no solution in the real-world scenario.

Is the fact the mapping isn't working a bug in the code, or am I using the wrong (version of) JsonSerDe. I'm using HDP-2.5.0.0

Hopefully someone can supply with an answer other then renaming the JSON field.

Thanks in advance..

1 ACCEPTED SOLUTION

avatar
Explorer

@Arjan Onwezen the hcatalog jsonSerde doesn't look like it uses the SERDEPROPERTIES mappings and only maps exact fieldnames to column names or JSON field names matching Hive internal column names, i.e _col0, _col1. I think the JsonSerde you are referring to is the rcongui Hive-JSON-Serde which should work as you've outlined.

View solution in original post

2 REPLIES 2

avatar
Explorer

@Arjan Onwezen the hcatalog jsonSerde doesn't look like it uses the SERDEPROPERTIES mappings and only maps exact fieldnames to column names or JSON field names matching Hive internal column names, i.e _col0, _col1. I think the JsonSerde you are referring to is the rcongui Hive-JSON-Serde which should work as you've outlined.

avatar
New Contributor

@Chris Cotter Thank you for our answer. I can confirm the RconGUI's version of JSON Serde works to circumvent the issue.