Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Jolt Transformation (nested array modifications)

avatar
Explorer

Hi everyone!

 

I hope all is well.

 

I have this json 

 

{
    "Data": {
        "ID": "09878666",
        "DATE": "2022-01-01",
        "ARRAY_ONE": [
            {
                "NAME": "test_1",
                "details": [
                    {
                        "address": "123123"
                    }
                ]
            },
            {
                "NAME": "test_2",
                "details": [
                    {
                        "address": "123123"
                    },
                    {
                        "address": "123123"
                    }
                ]
            }
        ]
    }
}

 

 

 

and I transformed the structure to this:

[
    {
        "ID": "09878666",
        "DATE": "2022-01-01",
        "NAME": "test_1",
        "details": [
            {
                "address": "123123"
            }
        ]
    },
    {
        "ID": "09878666",
        "DATE": "2022-01-01",
        "NAME": "test_2",
        "details": [
            {
                "address": "123123"
            },
            {
                "address": "123123"
            }
        ]
    }
]

 

Which is perfect except for the inner arrays, I want to flatten the array  to be like this

[
    {
        "ID": "09878666",
        "DATE": "2022-01-01",
        "NAME": "test_1",
        "details_0_adrress":"123123"
       
    },
    {
        "ID": "09878666",
        "DATE": "2022-01-01",
        "NAME": "test_2",
        "details_0_adrress":"123123",
        "details_1_adrress":"123123"
    }
]

 

 

Here is my spec:

[
    {
        "operation": "shift",
        "spec": {
            "Data": {
                "ARRAY_ONE": {
                    "*": {
                        "@(2,ID)": "[#2].ID",
                        "@(2,DATE)": "[#2].DATE",
                        "NAME": "[#2].NAME",
                        "details": "[#2].details"
                    }
                }
            }
        }
    }
]
 
 
 
 
I know how to flatten an array but I'm not sure how to do it using the current spec:,
What I would normally do is something like this:
{
        "operation": "shift",
        "spec": {
            "details": {
                "*": {
                    "address": "details_&1_address"
                }
            }
        }
    }
 
But it did not work so well.
 
 
Thank you!
2 ACCEPTED SOLUTIONS

avatar
Super Guru

Great job getting thus far with the Jolt spec, @Ibrahem !

 

Here's the remaining bit:

[
  {
    "operation": "shift",
    "spec": {
      "Data": {
        "ARRAY_ONE": {
          "*": {
            "@(2,ID)": "[#2].ID",
            "@(2,DATE)": "[#2].DATE",
            "NAME": "[#2].NAME",
            "details": {
              "*": {
                "address": "[#4].detail_&1_address"
              }
            }
          }
        }
      }
    }
  }
]

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

View solution in original post

avatar
Super Guru

Yes, it is possible:

[
  {
    "operation": "shift",
    "spec": {
      "Data": {
        "ARRAY_ONE": {
          "*": {
            "@(2,ID)": "[#2].ID",
            "@(2,DATE)": "[#2].DATE",
            "NAME": "[#2].NAME",
            "details": {
              "*": {
                "address": "[#4].detail_&1_address"
              }
            },
            "@(0,details)": "[#2].details"
          }
        }
      }
    }
  }
]
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

View solution in original post

4 REPLIES 4

avatar
Super Guru

Great job getting thus far with the Jolt spec, @Ibrahem !

 

Here's the remaining bit:

[
  {
    "operation": "shift",
    "spec": {
      "Data": {
        "ARRAY_ONE": {
          "*": {
            "@(2,ID)": "[#2].ID",
            "@(2,DATE)": "[#2].DATE",
            "NAME": "[#2].NAME",
            "details": {
              "*": {
                "address": "[#4].detail_&1_address"
              }
            }
          }
        }
      }
    }
  }
]

 

Cheers,

André

 

--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

Thank you so much @araujo for the help.

 

Is it possible to have the new flatten array and the original array within one jolt spec?

 

This is the current structure 

Ibrahem_0-1645530328805.png
 
can we change it to this:
Ibrahem_0-1645530156130.png

 

avatar
Super Guru

Yes, it is possible:

[
  {
    "operation": "shift",
    "spec": {
      "Data": {
        "ARRAY_ONE": {
          "*": {
            "@(2,ID)": "[#2].ID",
            "@(2,DATE)": "[#2].DATE",
            "NAME": "[#2].NAME",
            "details": {
              "*": {
                "address": "[#4].detail_&1_address"
              }
            },
            "@(0,details)": "[#2].details"
          }
        }
      }
    }
  }
]
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs up button.

avatar
Explorer

THANK YOU SO MUCH.

It makes total sense.