- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
NiFi PutHiveStreaming - NULL if hierarchical JSON is used
- Labels:
-
Apache NiFi
Created ‎03-23-2017 12:57 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Using the PutHiveStreaming processor and a verified correct AVRO schema for our data - NiFi puts only NULL if hierarchical JSON is used as input.
Example with our data below "Person"
{ "Person":{ "Name": "A", "Age": 23 } }
will produce NULL in the Hive table:
Name Age NULL NULL
While a "flat" JSON file, like:
{ "Name": "A", "Age": 23 }
will actually put the data into our hive table:
Name Age A 23
Does anyone have a solution so we can use JSON levels/hierarchies?
Thank you sincerely!
Created ‎03-23-2017 03:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As you suggested, you can use the JoltTransformJSON processor to "flatten" your JSON, but if your use case is as simple as the provided example (where the fields are one level down), you could also use EvaluateJsonPath with the target set to flowfile-content and the JSONPath expression set to $.Person
Created ‎03-23-2017 02:40 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
...currently we are using JOLT as to not confuse the hbase-processor with parent-child levels
Created ‎03-23-2017 03:46 PM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As you suggested, you can use the JoltTransformJSON processor to "flatten" your JSON, but if your use case is as simple as the provided example (where the fields are one level down), you could also use EvaluateJsonPath with the target set to flowfile-content and the JSONPath expression set to $.Person
