Skip to main content
pradeepraovenna
pradeepraovenna
Silver ⭐️⭐️
May 10, 2019
Solved

Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

I'm a newbie to Nifi and learning eventually. I'm looking to create a JSON array by utilizing 5 fields of 15 fields JSON request by using JoltTransformJSON processor. JSON array must ignore the fields if the value is null/empty. @Matt Burgess

Input:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us"

}


I'm trying to build JSON array of "color_* "attributes and current date parameter to followed by color. My output should consist of existing input + newly built JSON array which consists of 3 color as the color_2, color_4 are null.

Expected Output:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us",

"colorsLove" : [ {

"color_1" : "white",

"date" : "20190905"

}, {

"color_3" : "black",

"date" : "20190905"

}, {

"color_5" : "blue",

"date" : "20190905"

}]

}



mburgess
mburgessAnswer
Gold ⭐️⭐️⭐️
May 10, 2019

This Chain spec will add the hardcoded value 20190905 into the array (after removing empty values):

[
  {
    "operation": "shift",
    "spec": {
      "color_*": {
        "": "TRASH",
        "*": {
          "$": "colorsLove[].&2"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "colorsLove": {
        "*": {
          "#20190905": "colorsLove[#2].date",
          "*": "colorsLove[#2].&"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "remove",
    "spec": {
      "TRASH": ""
    }
  }
]


You should be able to replace "#20190905" with a NiFi Expression Language statement, maybe something like:

"#${now:toNumber():format('yyyyddMM')}"

... but I didn't try that part.

View in thread

4 replies

mburgess
Gold ⭐️⭐️⭐️
mburgessAnswer
Gold ⭐️⭐️⭐️
May 10, 2019

This Chain spec will add the hardcoded value 20190905 into the array (after removing empty values):

[
  {
    "operation": "shift",
    "spec": {
      "color_*": {
        "": "TRASH",
        "*": {
          "$": "colorsLove[].&2"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "colorsLove": {
        "*": {
          "#20190905": "colorsLove[#2].date",
          "*": "colorsLove[#2].&"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "remove",
    "spec": {
      "TRASH": ""
    }
  }
]


You should be able to replace "#20190905" with a NiFi Expression Language statement, maybe something like:

"#${now:toNumber():format('yyyyddMM')}"

... but I didn't try that part.

Translate
    mburgess
    Gold ⭐️⭐️⭐️
    mburgess
    Gold ⭐️⭐️⭐️
    May 11, 2019

    To address your comment below, I missed the part where you want to call the outgoing field "color". Change this line (8):

    "$": "colorsLove[].&2"

    To this:

    "$": "colorsLove[].color"


    Translate
      pradeepraovenna
      pradeepraovennaAuthor
      Silver ⭐️⭐️
      May 15, 2019

      Thank you so much Matt, this worked pretty well for me

      Translate
      pradeepraovenna
      pradeepraovennaAuthor
      Silver ⭐️⭐️
      May 11, 2019

      Hey @Matt Burgess - Thank you so much for responding, it perfectly worked as I needed. I've a slight change in output of JSON array.

      Output JSON Array should always contain "color_*" attribute as "color" always. Below is the sample output that I'm looking for.

      Expected Output:

      {

      "lastName" : "cgLastName6",

      "gender" : "Male",

      "currentDate" : "20190509",

      "middleInitial" : "D",

      "address_2" : "fake addy 2",

      "mailingZipCode" : "55555",

      "provider_address_1" : "123 fake addy",

      "color_1" : "white",

      "color_2" : "",

      "color_3" : "black",

      "color_4" : "",

      "color_5" : "blue",

      "homeAddress" : "123 fake st",

      "primaryLanguage" : "",

      "country" : "us",

      "colorsLove" : [ {

      "color" : "white",

      "date" : "20190905"

      }, {

      "color" : "black",

      "date" : "20190905"

      }, {

      "color" : "blue",

      "date" : "20190905"

      }]

      }

      Translate
      Terms & ConditionsAccessibility statement