How can I process different JSON responses and merge them?

[#1]

Hi, I have an API call to a service, that service have different type of processes, the API cannot filter them.

So when I query the API it returns JSON with 30 common keys, and another 15 that varies for each case, the question is, is there a way to automatically adapt the schema so I can use MergeRecords? The result goes to S3.

There are a lot of cases, and it seems they are adding more once in a while, is there a "generic" way to tell MergeRecords to "if you see any change, make another group", I don't have a key with value to determine the group. Any idea?

Thanks!

[#2]

Hey,

Can you provide us with a little more context on what the API is? When I quickly research on "MergeRecords", it shows about NiFi.

However, from the current understanding, I hope the URL [1] would be helpful for you.

Thanks!

[1] https://stackoverflow.com/a/55124212/4340139