Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Count number of incoming flow files

avatar
Explorer

I have to make a flow where I want to check that two Mongo DBs are in sync so everyday I have to compare the count of records that are being inserted into both the DBs if there is a discrepancy I want to transfer the records from the main DB to the DB which doesn't have it. Both DBs have a created AT key which has the date when the record was added. I want to achieve this through apache NIFI.  

1 REPLY 1

avatar

@Fanxxx,

How I would do the first POC:
1) GetMongoRecord: execute the count on the first table. Using the property "Query Output Attribute" you save that value directly as an attribute.
2) connected to the success queue another GetMongoRecord: execute the count on the second table. Using the property "Query Output Attribute" you save that value directly as an attribute. 
3) connected to the success queue an RouteOnAttribute: here you define a rules --> if count1=count2, do what you want to do, otherwise call the logic for the insert, as you said. (using NiFi Expression language: ${attribute1:equals(${attribute2})} )