Support Questions
Find answers, ask questions, and share your expertise

Monitor activity getting triggerd after start and stop

Monitor activity getting triggerd after start and stop

New Contributor

I need to trigger a mail after putting thousands of data using PutSQl.I kept a monitor activity with wait period 2 mints to trigger an email after completing data dumping .My problem now is monitor activity get triggerd after start stop of nifi..Any solutions?

1 REPLY 1

Re: Monitor activity getting triggerd after start and stop

Master Guru

@christy 

 

NiFi is primarily designed for continuous dataflow.  It sounds like your use case is for batch based processing flow.

The MonitorActivity processor was designed with the intent to notify when FlowFiles stop flowing down a continuous dataflow path.  Sounds like you are using the Lack of Activity relationship as a notifier for your batch job completion?  

On NiFi restart, the MonitorActivity will not have retained any state so the timer starts over and at 2 minutes of no activity will generate the lack of activity FlowFile again.

Thinking outside the box here, but you may want to consider the following alternative dataflow here:

1. After the putSQL has sent your FlowFileContent, consider using a ModifyBytes processor to remove the FlowFile's content since you no longer need it.
2. Connect "Success" relationship from ModifyBytes to a MergeContent processor. 
3. Configure the MergeContent processor to merge with "Minimum Number of Entries" set to a value high enough to exceed the number of FlowFiles that will be processed by the PutSQL.  Then set the "Max Bin Age" to 2 mins.  What this property does is tell MergeContent to merge a bin after 2 minutes regardless of the number of FlowFiles allocated to the bin.   Since we always want it to wait the full 2 minutes from the time the first FlowFile goes in to the bin, that is reason for high Min Num of entries value.
4. Connect the "Merged" relationship from the MergeContent to your putEmail processor.

The purpose of the ModifyBytes processor is to reduce CPU and disk I/O during the merge operation since there is no need to retain the content at this point in your dataflow.
The above flow will not trigger unwanted emails on a NiFi restart since the 2 minute timer does not start until the first FlowFile gets allocated to a bin.  

 

There may be other out of the box solutions other may offer as well, but this is just one.

 

Hope this helps,

Matt