question Re: Nifi putemail single notification alert for multiple files in Support Questions

Nifi putemail single notification alert for multiple files

Ash1 — Fri, 02 Jul 2021 05:34:08 GMT

I have a requirement to check the count of files in a directory (say 10) everyday after it is transferred and to send Just one email alert if it is not satisfied. I have to check this condition every day whether these 10 files have been transferred without any fail . Also my previous files Would not be deleted from the directory as well.
I am really new to nifi hence has a doubt the n how to proceed .
@MattWho Could you please help with this somehow

Re: Nifi putemail single notification alert for multiple files

SagarKanani — Mon, 05 Jul 2021 11:52:43 GMT

Hi...For a single email alert, use MergeContent processor (on some condition) before PutEmail. If you have a definite number of files then in MergeContent you can add Minimum Number of Entries as 10. If not, then only way is time bound.

For transfer check, you will have to provide more info here. Where is it transferred? Does nifi handle the transfer?

If nifi handles the transfer, you will have files in Failure relationship so that should tell you if the transfer for all files was successful or not.

If you find the answer helpful please accept this as a solution.

Re: Nifi putemail single notification alert for multiple files

Ash1 — Mon, 05 Jul 2021 14:54:05 GMT

Hey sagar , yes nifi handles the transfer of files but is there any way to check the count of files transferred to a folder successfully and to raise a failure alert if it fails in any way using nifi?

Re: Nifi putemail single notification alert for multiple files

SagarKanani — Tue, 06 Jul 2021 06:13:08 GMT

Hi..,if you know the exact number of files to be transferred then yes….but if the count is variable then since nifi is a flow tool there is no concept as the last file….

You can send an email if you find any files in the failure relationship… this way you will know if the transfer went well or not..,

If this seems obvious please share a sketch of what you wish to inplement for a better solution…

Re: Nifi putemail single notification alert for multiple files

MattWho — Tue, 13 Jul 2021 19:38:21 GMT

@Ash1

Not clear from yoru query how files are getting into or out of your NiFi.

Assuming you have already received a set fo FlowFiles for the day in to your NiFi dataflow, the best approach may be to notify if any fails to be written out /transferred at the end of your dataflow. In this manor not only would you know that not all Files transferred, but would know exact what file failed to transfer.

There are numerous processors that handle writing out FlowFile content (transfer) to another source or local file system. Those processing components typically have relationships for handling various types of failures. These relationships could be sent through a retry loop via the RetryFlowFile [1] processor back to the same transfer processor that failed. You define in the RetryFlowFile processor how many times you want a FlowFile to traverse this loop. After X number of loop it would get routed out of the loop to your PutEmail [2] processor where you could dynamically set the email content to include attributes from that FlowFile like filename, hostname of NiFi that failed to transfer it, etc...
From the PutEmail processor you could send that FlowFile to somewhere else for holding until manual intervention was taken in response to that email.

[`1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.RetryFlowFile/index.html

[2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.PutEmail/index.html

If you found any of the response given here assisted with your query, please take a moment to login and click "Accept" on each of those solutions.

Thank you,

Matt