Skip to main content
Dim
Dim
Gold ⭐️
August 25, 2023
Solved

NiFi MergeRecord change number double- int

Hi all. I can't figure out why the MergeRecord processor converts fractional values ​​to integers. Reading origin from json file.

Dim
DimAnswer
Gold ⭐️
August 29, 2023

Problem solved by setting Decimal() type in JoltTransformJSON"stake" processor: ${stake:toDecimal()}

View in thread

9 replies

Dim
DimAuthor
Gold ⭐️
August 25, 2023

Example :   "stake" : 0.5 --->    "stake" : 0  or 1

Translate
DianaTorres
Community Manager
DianaTorres
Community Manager
August 25, 2023

@Dim Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @cotopaul @MattWho  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.

Translate
Regards, Diana Torres, Senior Community Moderator
cotopaul
cotopaul
Platinum ⭐️
August 28, 2023

@Dim, I do not think that MergeRecord is the one doing this action, rather the schema you have defined in both your RecordReader and your RecordWriter.

 

I am working for example with streaming data in Parquet and AVRO Format, using MergeRecord three times during the flow and each fractional remains fractional, because I set the RecordWriter to have a schema which accepts fractional data.

I suggest you to take a look in the schema you have defined within your Controller Services and start debugging from there 🙂

Besides that, your problem might start from a different location. You need to check your flow from start to end and see if you are working with correct scale and precision and if the data types are correct or not.

Translate
Dim
DimAuthor
Gold ⭐️
August 28, 2023

222.jpg333.jpgThanks for the hint on the schema. I am using JsonTreeReader and JsonRecordSetWriter as standard. What schema options can be used to make MergeRecord work?

Translate
cotopaul
cotopaul
Platinum ⭐️
August 28, 2023

Well first of all, how does the data look like before entering MergeRecord?
Secondly, how did you configure both the Reader and the Writer? You pasted the configuration for MergeRecord, where this has nothing to do with how the data gets transformed.

Translate
Dim
DimAuthor
Gold ⭐️
August 28, 2023


I note that there are cases when the union occurs normally. But the dependency is not installed.

Translate
Dim
DimAuthorAnswer
Gold ⭐️
August 29, 2023

Problem solved by setting Decimal() type in JoltTransformJSON"stake" processor: ${stake:toDecimal()}

Translate
Terms & ConditionsAccessibility statement