Support Questions
Find answers, ask questions, and share your expertise

NIFI wait\notify operator when running scheduled FlowFile

NIFI wait\notify operator when running scheduled FlowFile

New Contributor

Hi there,

 

I'm new to NIFI and I've been working on what's becoming a pretty complicated data flow.  I'm currently using execute sql and then creating a timestamp for the last execution as my source db uses datetimeoffset for all date columns which doesn't seem to be supported by the QueryDatabase operator.  

 

My GenerateFlowFile is scheduled to run every thirty seconds.  I have wait and notify operators.  What I'm trying to do is only push data through to Couchbase once the value in the cache has been incremented.  I'm running into issues though where the same record is being inserted twice.  My guess is that the GenerateFlowFile executes and retrieves the same value from the cache before it's been updated.

 

I'm wondering how I can build my flow to avoid this?  Basically, how do I make sure my SQL doesn't execute using a variable which hasn't been updated?

 

Nathan_Smit_0-1617965976758.png