Support Questions

Find answers, ask questions, and share your expertise

NiFi Record Oriented Processor Filter

avatar
Expert Contributor

Let's say I have a JSON that looks similiar to this 

 

[{
"time": "",
"value": 92.72,
"longitude": "128.732855",
"latitude": "-16.1135755",
"gathered_time": "1685730397038",
"metadata": null
},
{
"time": "",
"value": 92.7,
"longitude": "128.732855",
"latitude": "-16.1135755",
"gathered_time": "1685730397038",
"metadata": null
},
{
"time": "",
"value": {
"nil": null
},
"longitude": "128.732855",
"latitude": "-16.1135755",
"gathered_time": "1685730397038",
"metadata": {
"TVPMeasurementMetadata": {
"qualifier": {
"title": null,
"href": null
}
}
}
}]

 How would I use a record oriented processor to filter out values that have a the key value equal to this -

{"value": {
"nil": null
}

In the example above it the last record in the JSON array would be filtered out so the expected output would look something similar to this - 

[{
"time": "",
"value": 92.72,
"longitude": "128.732855",
"latitude": "-16.1135755",
"gathered_time": "1685730397038",
"metadata": null
},
{
"time": "",
"value": 92.7,
"longitude": "128.732855",
"latitude": "-16.1135755",
"gathered_time": "1685730397038",
"metadata": null
}]

I don't understand how to do filters. I read about it in the RecordPath Guide but would love to see an example.

1 REPLY 1

avatar

@drewski7   The two posts below have solutions to query the json:


https://community.cloudera.com/t5/Community-Articles/Running-SQL-on-FlowFiles-using-QueryRecord-Proc...

 

https://community.cloudera.com/t5/Support-Questions/QueryRecord-processor-issue-with-nested-JSON/td-...


Here a the NiFI doc with more high level SQL info:

 

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/latest/org.apach...

 

You can find other examples here in the community just search something like: queryrecord + json.