Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

NiFi PutHiveStreaming - NULL if hierarchical JSON is used

Solved Go to solution
Highlighted

NiFi PutHiveStreaming - NULL if hierarchical JSON is used

New Contributor

Using the PutHiveStreaming processor and a verified correct AVRO schema for our data - NiFi puts only NULL if hierarchical JSON is used as input.

Example with our data below "Person"

{
"Person":{
	"Name": "A",
	"Age": 23
	}
}

will produce NULL in the Hive table:

Name	Age
NULL	NULL

While a "flat" JSON file, like:

{
	"Name": "A",
	"Age": 23
}

will actually put the data into our hive table:

Name	Age
A	23

Does anyone have a solution so we can use JSON levels/hierarchies?

Thank you sincerely!

1 ACCEPTED SOLUTION

Accepted Solutions

Re: NiFi PutHiveStreaming - NULL if hierarchical JSON is used

As you suggested, you can use the JoltTransformJSON processor to "flatten" your JSON, but if your use case is as simple as the provided example (where the fields are one level down), you could also use EvaluateJsonPath with the target set to flowfile-content and the JSONPath expression set to $.Person

2 REPLIES 2

Re: NiFi PutHiveStreaming - NULL if hierarchical JSON is used

New Contributor

...currently we are using JOLT as to not confuse the hbase-processor with parent-child levels

Re: NiFi PutHiveStreaming - NULL if hierarchical JSON is used

As you suggested, you can use the JoltTransformJSON processor to "flatten" your JSON, but if your use case is as simple as the provided example (where the fields are one level down), you could also use EvaluateJsonPath with the target set to flowfile-content and the JSONPath expression set to $.Person