Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

Solved Go to solution
Highlighted

Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

New Contributor

I'm a newbie to Nifi and learning eventually. I'm looking to create a JSON array by utilizing 5 fields of 15 fields JSON request by using JoltTransformJSON processor. JSON array must ignore the fields if the value is null/empty. @Matt Burgess

Input:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us"

}


I'm trying to build JSON array of "color_* "attributes and current date parameter to followed by color. My output should consist of existing input + newly built JSON array which consists of 3 color as the color_2, color_4 are null.

Expected Output:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us",

"colorsLove" : [ {

"color_1" : "white",

"date" : "20190905"

}, {

"color_3" : "black",

"date" : "20190905"

}, {

"color_5" : "blue",

"date" : "20190905"

}]

}



1 ACCEPTED SOLUTION

Accepted Solutions

Re: Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

This Chain spec will add the hardcoded value 20190905 into the array (after removing empty values):

[
  {
    "operation": "shift",
    "spec": {
      "color_*": {
        "": "TRASH",
        "*": {
          "$": "colorsLove[].&2"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "colorsLove": {
        "*": {
          "#20190905": "colorsLove[#2].date",
          "*": "colorsLove[#2].&"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "remove",
    "spec": {
      "TRASH": ""
    }
  }
]


You should be able to replace "#20190905" with a NiFi Expression Language statement, maybe something like:

"#${now:toNumber():format('yyyyddMM')}"

... but I didn't try that part.

4 REPLIES 4

Re: Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

This Chain spec will add the hardcoded value 20190905 into the array (after removing empty values):

[
  {
    "operation": "shift",
    "spec": {
      "color_*": {
        "": "TRASH",
        "*": {
          "$": "colorsLove[].&2"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "shift",
    "spec": {
      "colorsLove": {
        "*": {
          "#20190905": "colorsLove[#2].date",
          "*": "colorsLove[#2].&"
        }
      },
      "*": "&"
    }
  },
  {
    "operation": "remove",
    "spec": {
      "TRASH": ""
    }
  }
]


You should be able to replace "#20190905" with a NiFi Expression Language statement, maybe something like:

"#${now:toNumber():format('yyyyddMM')}"

... but I didn't try that part.

Re: Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

To address your comment below, I missed the part where you want to call the outgoing field "color". Change this line (8):

"$": "colorsLove[].&2"

To this:

"$": "colorsLove[].color"


Re: Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

New Contributor

Thank you so much Matt, this worked pretty well for me

Re: Nifi - Jolt Transform JSON Processor - Create a JSON array by utilizing few fields of exiting JSON and ignore the null attributes while constructing the array

New Contributor

Hey @Matt Burgess - Thank you so much for responding, it perfectly worked as I needed. I've a slight change in output of JSON array.

Output JSON Array should always contain "color_*" attribute as "color" always. Below is the sample output that I'm looking for.

Expected Output:

{

"lastName" : "cgLastName6",

"gender" : "Male",

"currentDate" : "20190509",

"middleInitial" : "D",

"address_2" : "fake addy 2",

"mailingZipCode" : "55555",

"provider_address_1" : "123 fake addy",

"color_1" : "white",

"color_2" : "",

"color_3" : "black",

"color_4" : "",

"color_5" : "blue",

"homeAddress" : "123 fake st",

"primaryLanguage" : "",

"country" : "us",

"colorsLove" : [ {

"color" : "white",

"date" : "20190905"

}, {

"color" : "black",

"date" : "20190905"

}, {

"color" : "blue",

"date" : "20190905"

}]

}