question Re: JsonRecordSetWriter 1.22.0 issues with elements in array in Support Questions

JsonRecordSetWriter 1.22.0 issues with elements in array

SteliosCh — Thu, 24 Aug 2023 08:24:31 GMT

While working with JsonRecordSetWriter 1.22.0 on an ExecuteSQLRecord 1.22.0 processor, we have noticed that when selecting columns with arrays such as the below example:

the writer has the below output:

However, if I select a smaller number in the array:

I seem to get the correct result:

This seems to be an overflow of some kind that did not seem to exist on our previous version of nifi when working with nifi 1.11.4

I am providing below both the settings of the ExecuteSQLRecord processors and the JsonRecordSetWriter. Please advise if this is a bug or If there might be something I am overlooking. Thank you.

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

VidyaSargur — Thu, 24 Aug 2023 09:21:08 GMT

@SteliosCh, Welcome to our community! To help you get the best possible answer, I have tagged in our NiFi experts @cotopaul @SAMSAL @TimothySpann @MattWho may be able to assist you further.

Please feel free to provide any additional information or details about your query, and we hope that you will find a satisfactory solution to your question.

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

SteliosCh — Thu, 24 Aug 2023 09:26:59 GMT

Thank you very much!

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

TimothySpann — Thu, 24 Aug 2023 13:41:34 GMT

I wonder if this was fixed 1.23.2. That seems like it shouldn't be too big, but it may be close.

https://calcite.apache.org/docs/reference.html

INTEGER, INT	4 byte signed integer	Range is -2147483648 to 2147483647

try BIGINT

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

SteliosCh — Thu, 24 Aug 2023 13:59:03 GMT

Hello @TimothySpann thank you for the reply,

Unfortunately BIGINT yields the same results

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

TimothySpann — Thu, 24 Aug 2023 14:22:16 GMT

What database is that for, that is very weird SQL syntax, why are you making it JSON to convert to int?

Trying upping the precision from 10 to 14.

Default Decimal Precision

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

SteliosCh — Thu, 24 Aug 2023 14:53:35 GMT

Changing decimal precision did not affect the results either. Could this be an issue with the writer by any chance?

This is a postgress database. The casting into json is to create a column of type array with integer contents.

Re: JsonRecordSetWriter 1.22.0 issues with elements in array

opqi — Thu, 08 Aug 2024 08:29:00 GMT

@SteliosCh, are there any updates on the issue?