question Re: How can I adjust my JOLT transformation to force an output property to be a string in Archives of Support Questions (Read Only)

How can I adjust my JOLT transformation to force an output property to be a string

dave_sargrad — Tue, 30 Oct 2018 19:59:53 GMT

The following code is the current JOLT transformation that I am using in a NIFI JoltTransformJSON processor:

[
  {
    "operation": "shift",
    "spec": {
      "textNOTAM": {
        "NOTAM": {
          // simple match.  Put the value '4' in the output under the "Rating" field
          "id": "&0",
          "number": "&0",
          "year": "&0",
          "type": "&0",
          "issued": "&0",
          "affectedFIR": "&0",
          "minimumFL": "&0",
          "maximumFL": "&0",
          "radius": "&0",
          "location": "&0",
          "effectiveStart": "&0",
          "effectiveEnd": "&0",
          "schedule": "&0",
          "text": "&0"
        }
      }
    }
}]

A typical flow file coming out of my JOLT transformation looks like this:

{"id":"NOTAM_1_51227850","number":376,"year":2018,"type":"N","issued":"2018-10-30T12:29:00.000Z","affectedFIR":"LIMM","minimumFL":"000","maximumFL":999,"radius":5,"location":"LIPA","effectiveStart":201810301229,"effectiveEnd":201811032359,"text":"NON-STANDARD BARRIER CONFIGURATION. BARRIER 2 (BAK-12) UNSERVICEABLE"}

The problem I am having is with the effectiveStart and effectiveEnd fields. I need the value to be a string, not a number. How can I adjust the JOLT transformation above to ensure that it is a string?

The JSON that comes out of this processor, I then inject into MONGO using a PutMongo processor.

Because of this input format problem, with effectiveStart and effectiveEnd, they are coming out of mongo as follows:

{ "_id" : ObjectId("5bc8b2808aad6206ababaa4b"), "id" : "NOTAM_1_51105982", "number" : 7, "year" : 2018, "type" : "N", "issued" : "2018-10-16T00:13:00.000Z", "affectedFIR" : "ZBW", "minimumFL" : "000", "maximumFL" : 999, "radius" : 5, "location" : "OXC", "effectiveStart" : NumberLong("201810160013"), "effectiveEnd" : "201810312359EST", "text" : "SVC AUTOMATED WX BCST SYSTEM PRECIPITATION OUT OF SERVICE" }

Notice that the effectiveStart is a NumberLong. How can I adjust the JOLT transform to ensure that the effectiveStart and effectiveEnd are seen as strings when the JSON flow file is sent to PutMongo?

Re: How can I adjust my JOLT transformation to force an output property to be a string

mburgess — Tue, 30 Oct 2018 22:43:37 GMT

Do you need the numbers as-is into strings, or a formatted timestamp string, or something else?

Re: How can I adjust my JOLT transformation to force an output property to be a string

mburgess — Tue, 30 Oct 2018 22:45:33 GMT

Try the following chain spec (it uses Modify-overwrite instead of Shift):

[
  {
    "operation": "modify-overwrite-beta",
    "spec": {
      "effectiveStart": "=toString",
      "effectiveEnd": "=toString"
    }
  }
]

That will preserve all other fields, but change the values of effectiveStart/End to strings

Re: How can I adjust my JOLT transformation to force an output property to be a string

dave_sargrad — Wed, 31 Oct 2018 22:28:55 GMT

Thank you @Matt Burgess for the key operation that i was missing. Keep in mind, I still need the shift operation since I am reducing a more complex JSON as part of the shift

So the entire spec contains two operations as follows..

[
  {
    "operation": "shift",
    "spec": {
      "textNOTAM": {
        "NOTAM": {
          
          "id": "&0",
          "number": "&0",
          "year": "&0",
          "type": "&0",
          "issued": "&0",
          "affectedFIR": "&0",
          "minimumFL": "&0",
          "maximumFL": "&0",
          "radius": "&0",
          "location": "&0",
          "effectiveStart": "&0",
          "effectiveEnd": "&0",
          "schedule": "&0",
          "text": "&0"
        }
      }
    }
},
{
    "operation": "modify-overwrite-beta",
    "spec": {
      "effectiveStart": "=toString",
      "effectiveEnd": "=toString"
    }
  }
]