Created 01-30-2023 11:28 AM
I have a JSON dataset and need to create a unique key from it. No single field is unique, but the first two, together, are. What I want to convert from:
[{"airport":"KDEN", "runway":"16R", "length":16000}, ...]
and I want to concatenate the "airport" and "runway" values to create this:
[{"key":"KDEN/16R", "airport":"KDEN", "runway":"16R", "length":16000}, ...]
I've got it working with an ugly anti-pattern of SplitRecord, EvaluateJSONPath, UpdateAttribute, and AttributesToJSON which changes each flow file to several thousand, but am looking for a better way to do this.
Created 01-30-2023 02:34 PM
Hi,
There are multiple ways you can resolve this issue without having to do all processors you mentioned above. Some options:
1- QueryRecord
2- UpdateRecord
3- JsonJoltTransformation.
If you elect to use QueryRecord which is what is been suggested for such anti-pattern , then you can create new dynamic property on the QueryRecord to represent the new relationship which will have the result of the select statement with new Key field as follows :
The select query syntax for the "AddKeyRel":
select *,airport||'/'||runway as key from FlowFile
The || symbol is used for concatenation in such query and processor.
The JsonTreeReader and the JsonRecordSetWriter are configured as follows:
If the helps please accept solution.
Thanks
Created 01-30-2023 02:34 PM
Hi,
There are multiple ways you can resolve this issue without having to do all processors you mentioned above. Some options:
1- QueryRecord
2- UpdateRecord
3- JsonJoltTransformation.
If you elect to use QueryRecord which is what is been suggested for such anti-pattern , then you can create new dynamic property on the QueryRecord to represent the new relationship which will have the result of the select statement with new Key field as follows :
The select query syntax for the "AddKeyRel":
select *,airport||'/'||runway as key from FlowFile
The || symbol is used for concatenation in such query and processor.
The JsonTreeReader and the JsonRecordSetWriter are configured as follows:
If the helps please accept solution.
Thanks
Created 02-01-2023 02:27 PM
That worked perfectly. Thank you.