question Re: JSON object must always be treated as an array in Support Questions

JSON object must always be treated as an array

MarioFRS — Wed, 12 Oct 2022 11:14:34 GMT

Hello everyone.

i need help. I have a JSON file which contains in an object either an array of other objects or a single object. This is the relevant object "Product": [ or "Product": { .

How can I configure with the JOLT Transform processor to always expect an array in the object? The product object must always be treated as an array, even if it contains only a single object.

{ "costc": 9638, "sum_amount": 543, "sum_invitation": 0, "sum_selfconsumption": 0, "sum_loss": 0, "sum_gross": "2764,000", "sum_net": "2392,480", "sum_margin": "2003,780", "sum_costs_total": "388,700", "sum_costs": "388,700", "sum_costs_invitation": 0, "sum_costs_selfconsumption": 0, "sum_costs_loss": 0, "sum_costs_total_percent": "16,25", "sum_costs_percent": "16,25", "Classes": { "ClassSummary": [ { "class": 1102, "sum_amount": 117, "sum_invitation": 0, "sum_selfconsumption": 0, "sum_loss": 0, "sum_gross": "740,500", "sum_net": "692,060", "sum_margin": "552,385", "sum_costs_total": "139,675", "sum_costs": "139,675", "sum_costs_invitation": 0, "sum_costs_selfconsumption": 0, "sum_costs_loss": 0, "sum_costs_total_percent": "20,18", "sum_costs_percent": "20,18", "ProductDetails": { "Product": [ { "id": 7992160, "artnr": 32212, "sum_amount": 16, "sum_invitation": 0, "sum_selfconsumption": 0, "sum_loss": 0, "sum_gross": "80,000", "sum_net": "74,770", "sum_margin": "53,243", "sum_costs_total": "21,527", "sum_costs": "21,527", "sum_costs_invitation": 0, "sum_costs_selfconsumption": 0, "sum_costs_loss": 0, "sum_costs_total_percent": "28,79", "sum_costs_percent": "28,79", "value": null }, { "id": 7794909, "artnr": 32205, "sum_amount": 7, "sum_invitation": 0, "sum_selfconsumption": 0, "sum_loss": 0, "sum_gross": "56,000", "sum_net": "52,340", "sum_margin": "39,473", "sum_costs_total": "12,867", "sum_costs": "12,867", "sum_costs_invitation": 0, "sum_costs_selfconsumption": 0, "sum_costs_loss": 0, "sum_costs_total_percent": "24,58", "sum_costs_percent": "24,58", "value": null }, { "class": 19174, "sum_amount": 17, "sum_invitation": 0, "sum_selfconsumption": 0, "sum_loss": 0, "sum_gross": "85,000", "sum_net": "71,430", "sum_margin": "71,430", "sum_costs_total": 0, "sum_costs": 0, "sum_costs_invitation": 0, "sum_costs_selfconsumption": 0, "sum_costs_loss": 0, "sum_costs_total_percent": 0, "sum_costs_percent": 0, "ProductDetails": { "Product": { "id": 7925180, "artnr": 6201, "sum_amount": 17, "sum_invitation": 0, "sum_selfconsumption": 0, "sum_loss": 0, "sum_gross": "85,000", "sum_net": "71,430", "sum_margin": "71,430", "sum_costs_total": 0, "sum_costs": 0, "sum_costs_invitation": 0, "sum_costs_selfconsumption": 0, "sum_costs_loss": 0, "sum_costs_total_percent": 0, "sum_costs_percent": 0, "value": null } } } ] } } ] }, "date": "2022-10-06" }

Re: JSON object must always be treated as an array

ChuckE — Wed, 12 Oct 2022 22:37:11 GMT

Try this...

[ { "operation": "cardinality", "spec": { "*": { "*": { "Product": "MANY" } } } } ]

Re: JSON object must always be treated as an array

MarioFRS — Thu, 13 Oct 2022 06:26:04 GMT

@ChuckEIt does not work. I still get the same output as the input is. The single object is not yet treated as an array. Do you have another idea

Re: JSON object must always be treated as an array

ChuckE — Thu, 13 Oct 2022 08:40:04 GMT

"Product" is nested several layers deep so the first thing you need to do is match the wildcard (*) character to match the depth of where "Product" is located. As a simplified example, I trimmed and flattened your JSON data down a little, and the transform works as expected. Where I ran into a challenge was with the "ClassSummary" array, because I don't know how to account for it in the structure. Hopefully this gives you something to work with, and using this web site you get some ideas on how to take it further. I'm very curious to see the result.

https://jolt-demo.appspot.com/#ritwickgupta

[ { "operation": "cardinality", "spec": { "*": { "*": { "Product": "MANY" } } } } ]

{ "costc": 9638, "sum_amount": 543, "Classes": { "class": 1102, "sum_amount": 117, "sum_invitation": 0, "ProductDetails": { "Product": { "id": 7992160, "artnr": 32212, "sum_amount": 16, "sum_invitation": 0, "value": null } } }, "date": "2022-10-06" }

Re: JSON object must always be treated as an array

MarioFRS — Thu, 13 Oct 2022 11:57:47 GMT

@ChuckEThanks for your explanation. I had also tested it with a simplified json file and it worked there. I have replaced the expected wildcards with the names of the objects to clarify the structure. And I still have no idea why it does not work

[ { "operation": "cardinality", "spec": { "*": { "ClassSummary": { "ProductDetails": { "Product": "MANY" } } } } } ]

Re: JSON object must always be treated as an array

ChuckE — Thu, 13 Oct 2022 15:33:21 GMT

This one works. Give it a try...

[ { "operation": "cardinality", "spec": { "*": { "*": { "ClassSummary": "MANY", "*": { "*": { "Product": "MANY" } } } } } } ]

Re: JSON object must always be treated as an array

MarioFRS — Thu, 13 Oct 2022 18:49:45 GMT

This works not with a combination of single object and an array of objects for "product". It works for me only with a single object. For example

Re: JSON object must always be treated as an array

ChuckE — Thu, 13 Oct 2022 19:26:06 GMT

Hmm. I tried all the use cases (single object, array, no array) and this transform works with all the example data that you've posted. Can you send a screen shot of what you mean; perhaps I don't understand the problem.

Re: JSON object must always be treated as an array

MarioFRS — Thu, 13 Oct 2022 19:54:59 GMT

I will try it. As you can see in the example, it can happen that a class within the classes summary, object product details can have an array of objects, or a single object.

As you can see in the example, it can happen that a class within the classes Summary, Object Product Details can have an array of objects or a single object.

And the Jolt Spec has no influence that a single object becomes an array.

Re: JSON object must always be treated as an array

ChuckE — Thu, 13 Oct 2022 21:16:17 GMT

Based on this screen shot it appears you are using the wrong Jolt Transform. Try using the newest version that I sent earlier this morning.

I've also attached screen shots which show the Jolt works with your data. I trimmed your data up a little for brevity purposes, but the structure is still the same.

Re: JSON object must always be treated as an array

ChuckE — Fri, 14 Oct 2022 23:23:32 GMT

Hey @MarioFRS, Just following up to make sure things are good to go. If you are all set then please accept the solution to ensure it helps others in the future.

Re: JSON object must always be treated as an array

MarioFRS — Mon, 17 Oct 2022 07:03:57 GMT

Hey @ChuckE Sorry for the delay. It doesn't work for me, I'm using the latest Jolt specification you provided and the entire json file, with individual objects and the array of objects from product. The single product object in a class does not become an array

Re: JSON object must always be treated as an array

ChuckE — Mon, 17 Oct 2022 18:57:41 GMT

@MarioFRS Ah, you are correct. I overlooked that additional nested Product. That complicates things because the upper Product can be an array or not any array, which itself contains another ProductDetails.Product which also may or may not be an array. Ugh!

To solve this I had to resort to chaining multiple transforms to account for all the possible formats that can exist. I may not have all the possible permutations you'll have, but you can follow the basic structure of these transforms and create new ones with additional levels if necessary.

It is important to note that the inner-most portions of the nested data need transformed first, so the order of the transforms starts with the inner-most data and ends with the outer-most data. Keep that in mind if you need to add more transforms to account for additional permutations of the data.

Give this a try and let me know how it works for you.

[ { "operation": "cardinality", "spec": { "*": { "*": { "*": { "*": { "*": { "*": { "*": { "Product": "MANY" } } } } } } } } }, { "operation": "cardinality", "spec": { "*": { "*": { "*": { "*": { "*": { "*": { "Product": "MANY" } } } } } } } }, { "operation": "cardinality", "spec": { "*": { "*": { "*": { "*": { "Product": "MANY" } } } } } }, { "operation": "cardinality", "spec": { "*": { "*": { "*": { "Product": "MANY" } } } } }, { "operation": "cardinality", "spec": { "*": { "*": { "ClassSummary": "MANY" } } } }, { "operation": "cardinality", "spec": { "*": { "ClassSummary": "MANY" } } } ]

Re: JSON object must always be treated as an array

MarioFRS — Tue, 18 Oct 2022 06:23:05 GMT

@ChuckE that's it. Thanks for your support in this case. Now, it works fine.