Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Record duplication in MongoDB

avatar
Explorer

I have a mongo aggregation query which runs in "RunMongoAggregation" Processor. from using that BULK RESULT, inserted to the mongo collection. For the insertion I'm using "PutMongoRecord" in NIFI 1.25.0 at once. But the issue is this,
I need to run this mongo aggregation query in time to time. When it runs in time to time, data will duplicated. It has a Unique Id call "PortfolioTransactionID". 
here is the sample json,


{
"PortfolioBaseID": "371",
"PortfolioBaseIDOrder": 37539,
"PortfolioTransactionID": 1416807,
"RecallableDistributedGainLocal": null,
"SecTypeCode1": "mf",
"SecTypeCode2": "ca",
"SplitFactor": 1,
"SpotRate": null,
"ThruDate": "2016-12-28",
"TradeAmount": "5000000.0",
"TradeAmountLocal": 5000000.0
}

How do I avoid that data duplication using "PortfolioTransactionID" unique key while insert data as bulk.. I'm using NiFi 1.25.0

Thanks.

0 REPLIES 0