Concatenate two JSON values

Question

I have a JSON dataset and need to create a unique key from it. No single field is unique, but the first two, together, are. What I want to convert from:

[{"airport":"KDEN", "runway":"16R", "length":16000}, ...]

and I want to concatenate the "airport" and "runway" values to create this:

[{"key":"KDEN/16R", "airport":"KDEN", "runway":"16R", "length":16000}, ...]

I've got it working with an ugly anti-pattern of SplitRecord, EvaluateJSONPath, UpdateAttribute, and AttributesToJSON which changes each flow file to several thousand, but am looking for a better way to do this.

SAMSAL · Accepted Answer

Hi,

There are multiple ways you can resolve this issue without having to do all processors you mentioned above. Some options:

1- QueryRecord

2- UpdateRecord

3- JsonJoltTransformation.

If you elect to use QueryRecord which is what is been suggested for such anti-pattern , then you can create new dynamic property on the QueryRecord to represent the new relationship which will have the result of the select statement with new Key field as follows :

The select query syntax for the "AddKeyRel":

select *,airport||'/'||runway as key from FlowFile

The || symbol is used for concatenation in such query and processor.

The JsonTreeReader and the JsonRecordSetWriter are configured as follows:

If the helps please accept solution.

Thanks

Sign up

Please use your Cloudera account.

Login to the community

Please use your Cloudera account.