question Re: Filter and removing using jolt in Support Questions

Filter and removing using jolt

donaldo71 — Mon, 10 Feb 2025 11:21:11 GMT

Hi i have json input like that:

{
  "messageId": 1234,
  "markerId": "T",
  "dateFrom": 6436058131202690000,
  "dateTo": -3840351829778683400,
  "records": [
    {
      "recordId": 1,
      "account": "152739203233"
    },
    {
      "recordId": 2,
      "data": {
        "email": "jsmith@gmail.com",
        "firstName": "John",
        "lastName": "Smith"
      }
    },
    {
      "recordId": 3,
      "city": "Los Angeles"
    },
    {
      "recordId": 4,
      "idNumber": "12345"
    },
    {
      "recordId": 5,
      "accountNumber": "55671"
    },
    {
      "recordId": 6,
      "account": "6789189790191"
    },
    {
      "recordId": 7,
      "city": "San Fransisco"
    }
  ]
}

And I would like to have output like that:

[ {
  "messageId" : 1234,
  "markerId" : "T"
  "dateFrom" : 6436058131202690000,
  "dateTo" : -3840351829778683400,
  "recordId" : 1,
 "account": "152739203233"
}, {
  "messageId" : 1234,
  "markerId" : "T"
  "dateFrom" : 6436058131202690000,
  "dateTo" : -3840351829778683400,
  "recordId" : 2,
  "email": "jsmith@gmail.com",
  "firstName": "John",
  "lastName": "Smith"
}, {
 "messageId" : 1234,
  "markerId" : "T"
  "dateFrom" : 6436058131202690000,
  "dateTo" : -3840351829778683400,
  "recordId" : 3,
  "city": "Los Angeles"
}, {
  "messageId" : 1234,
  "markerId" : "T"
  "dateFrom" : 6436058131202690000,
  "dateTo" : -3840351829778683400,
  "recordId" : 6,
  "account": "6789189790191"
}, {
   "messageId" : 1234,
  "markerId" : "T"
  "dateFrom" : 6436058131202690000,
  "dateTo" : -3840351829778683400,
  "recordId" : 7,
   "city": "San Fransisco"
} ]

i need to have only this record with account, data(email,firstName,lastName) or city, i dont need records with idNumber and accountNumber. Additionally, I need each record to have a common part: messageId, markerId, dateFrom and dateTo Is it possible to do something like that with jolt transformation?

Re: Filter and removing using jolt

donaldo71 — Mon, 10 Feb 2025 18:08:11 GMT

Hi again,

I managed how to split records into individual records thanks to JOLT like this:

[
{
"operation": "shift",
"spec": {
"records": {
"*": {
"@(2,messageId)": "[&1].messageId",
"@(2,markerId)": "[&1].markerId",
"@(2,dateFrom)": "[&1].dateFrom",
"@(2,dateTo)": "[&1].dateTo",
"recordId": "[&1].recordId",
"account": "[&1].account",
"data": {
"email": "[&2].email",
"firstName": "[&2].firstName",
"lastName": "[&2].lastName"
},
"city": "[&1].city"
}
}
}
}
]

Now my output is like this:

[ {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 1,
"account" : "152739203233"
}, {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 2,
"email" : "jsmith@gmail.com",
"firstName" : "John",
"lastName" : "Smith"
}, {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 3,
"city" : "Los Angeles"
}, {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 4
}, {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 5
}, {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 6,
"account" : "6789189790191"
}, {
"messageId" : 1234,
"markerId" : "T",
"dateFrom" : 6436058131202690000,
"dateTo" : -3840351829778683400,
"recordId" : 7,
"city" : "San Fransisco"
} ]

But still I dont now how to remove/filter records which have idNumber and accountNumber fields(in this case records 4,5,6). Someone can help me?