Created 10-30-2018 12:59 PM
The following code is the current JOLT transformation that I am using in a NIFI JoltTransformJSON processor:
[ { "operation": "shift", "spec": { "textNOTAM": { "NOTAM": { // simple match. Put the value '4' in the output under the "Rating" field "id": "&0", "number": "&0", "year": "&0", "type": "&0", "issued": "&0", "affectedFIR": "&0", "minimumFL": "&0", "maximumFL": "&0", "radius": "&0", "location": "&0", "effectiveStart": "&0", "effectiveEnd": "&0", "schedule": "&0", "text": "&0" } } } }]
A typical flow file coming out of my JOLT transformation looks like this:
{"id":"NOTAM_1_51227850","number":376,"year":2018,"type":"N","issued":"2018-10-30T12:29:00.000Z","affectedFIR":"LIMM","minimumFL":"000","maximumFL":999,"radius":5,"location":"LIPA","effectiveStart":201810301229,"effectiveEnd":201811032359,"text":"NON-STANDARD BARRIER CONFIGURATION. BARRIER 2 (BAK-12) UNSERVICEABLE"}
The problem I am having is with the effectiveStart and effectiveEnd fields. I need the value to be a string, not a number. How can I adjust the JOLT transformation above to ensure that it is a string?
The JSON that comes out of this processor, I then inject into MONGO using a PutMongo processor.
Because of this input format problem, with effectiveStart and effectiveEnd, they are coming out of mongo as follows:
{ "_id" : ObjectId("5bc8b2808aad6206ababaa4b"), "id" : "NOTAM_1_51105982", "number" : 7, "year" : 2018, "type" : "N", "issued" : "2018-10-16T00:13:00.000Z", "affectedFIR" : "ZBW", "minimumFL" : "000", "maximumFL" : 999, "radius" : 5, "location" : "OXC", "effectiveStart" : NumberLong("201810160013"), "effectiveEnd" : "201810312359EST", "text" : "SVC AUTOMATED WX BCST SYSTEM PRECIPITATION OUT OF SERVICE" }
Notice that the effectiveStart is a NumberLong. How can I adjust the JOLT transform to ensure that the effectiveStart and effectiveEnd are seen as strings when the JSON flow file is sent to PutMongo?
Created 10-30-2018 03:45 PM
Try the following chain spec (it uses Modify-overwrite instead of Shift):
[ { "operation": "modify-overwrite-beta", "spec": { "effectiveStart": "=toString", "effectiveEnd": "=toString" } } ]
That will preserve all other fields, but change the values of effectiveStart/End to strings
Created 10-30-2018 03:43 PM
Do you need the numbers as-is into strings, or a formatted timestamp string, or something else?
Created 10-30-2018 03:45 PM
Try the following chain spec (it uses Modify-overwrite instead of Shift):
[ { "operation": "modify-overwrite-beta", "spec": { "effectiveStart": "=toString", "effectiveEnd": "=toString" } } ]
That will preserve all other fields, but change the values of effectiveStart/End to strings
Created 10-31-2018 03:28 PM
Thank you @Matt Burgess for the key operation that i was missing. Keep in mind, I still need the shift operation since I am reducing a more complex JSON as part of the shift
So the entire spec contains two operations as follows..
[ { "operation": "shift", "spec": { "textNOTAM": { "NOTAM": { "id": "&0", "number": "&0", "year": "&0", "type": "&0", "issued": "&0", "affectedFIR": "&0", "minimumFL": "&0", "maximumFL": "&0", "radius": "&0", "location": "&0", "effectiveStart": "&0", "effectiveEnd": "&0", "schedule": "&0", "text": "&0" } } } }, { "operation": "modify-overwrite-beta", "spec": { "effectiveStart": "=toString", "effectiveEnd": "=toString" } } ]