Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

[#1]

I'm a newbie to Nifi and learning eventually. I'm looking to create a JSON array by utilizing 5 fields of 15 fields JSON request by using JoltTransformJSON processor. JSON array must ignore the fields if the value is null/empty. @Matt Burgess

Input:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us"

}


I'm trying to build JSON array of "color_* "attributes and current date parameter to followed by color. My output should consist of existing input + newly built JSON array which consists of 3 color as the color_2, color_4 are null.

Expected Output:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us",

"colorsLove" : [ {

"color_1" : "white",

"date" : "20190905"

}, {

"color_3" : "black",

"date" : "20190905"

}, {

"color_5" : "blue",

"date" : "20190905"

}]

}

[#2 · Accepted solution]

This Chain spec will add the hardcoded value 20190905 into the array (after removing empty values):

[
  {
    "operation": "shift",
    "spec": {
      "color_*": {
        "": "TRASH",
        "*": {
          "$": "colorsLove[].&2"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "colorsLove": {
        "*": {
          "#20190905": "colorsLove[#2].date",
          "*": "colorsLove[#2].&"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "remove",
    "spec": {
      "TRASH": ""
    }
  }
]

You should be able to replace "#20190905" with a NiFi Expression Language statement, maybe something like:

"#${now:toNumber():format('yyyyddMM')}"

... but I didn't try that part.

[#3]

To address your comment below, I missed the part where you want to call the outgoing field "color". Change this line (8):

"$": "colorsLove[].&2"

To this:

"$": "colorsLove[].color"

[#4]

Thank you so much Matt, this worked pretty well for me

[#5]

Hey @Matt Burgess - Thank you so much for responding, it perfectly worked as I needed. I've a slight change in output of JSON array.

Output JSON Array should always contain "color_*" attribute as "color" always. Below is the sample output that I'm looking for.

Expected Output:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us",

"colorsLove" : [ {

"color" : "white",

"date" : "20190905"

}, {

"color" : "black",

"date" : "20190905"

}, {

"color" : "blue",

"date" : "20190905"

}]

}