I have a mongo aggregation query which runs in "RunMongoAggregation" Processor. from using that BULK RESULT, inserted to the mongo collection. For the insertion I'm using "PutMongoRecord" in NIFI 1.25.0 at once. But the issue is this,
I need to run this mongo aggregation query in time to time. When it runs in time to time, data will duplicated. It has a Unique Id call "PortfolioTransactionID".
here is the sample json,
{
"PortfolioBaseID": "371",
"PortfolioBaseIDOrder": 37539,
"PortfolioTransactionID": 1416807,
"RecallableDistributedGainLocal": null,
"SecTypeCode1": "mf",
"SecTypeCode2": "ca",
"SplitFactor": 1,
"SpotRate": null,
"ThruDate": "2016-12-28",
"TradeAmount": "5000000.0",
"TradeAmountLocal": 5000000.0
}
How do I avoid that data duplication using "PortfolioTransactionID" unique key while insert data as bulk.. I'm using NiFi 1.25.0
Thanks.