Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Editing Nifi queue

Highlighted

Editing Nifi queue

New Contributor

Hi All, I am receiving flow files in failures and then sending it to PutEmail. Now the problem is if I receive 100 failures, I will be receiving 100 mails. So instead I want to be such that I receive the first mail with the first flow file, then waiting for 30 mins and receive mail again but with the last flow file of during the 30 mins time interval instead. 

3 REPLIES 3
Highlighted

Re: Editing Nifi queue

@ujay The solution you are looking for is the DetectDuplicate processor:

 

https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.5.0/org.apache...

 

This processor is used with Distributed Map Cache Client and Server (Controller Services) to deduplicate a flow based on your criteria.  I have a template demo here:

 

https://github.com/steven-matison/NiFi-Templates/blob/master/DetectDuplicate_DistributedMapCache_Dem...

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven @ DFHZ

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Highlighted

Re: Editing Nifi queue

New Contributor

Hi @stevenmatison

Thanks for the reply, can you tell me how to ingest your xml file in order to take reference, or if you could share a snap.

Thanks again!

 

Re: Editing Nifi queue

@ujay Of course.   The link referenced xml is a template file.  Click through and get the raw xml code and save to a file.  From here you import the template:

Screen Shot 2020-09-30 at 9.57.02 AM.png

Screen Shot 2020-09-30 at 9.57.25 AM.png

 

Then in the upper navigation grab the template icon and drag to the nifi canvas:

 

 

Screen Shot 2020-09-30 at 9.57.39 AM.png 

It should automatically choose the last template uploaded:

Screen Shot 2020-09-30 at 9.57.46 AM.png

 

Once the template is on the canvas click through into the process group created:

Screen Shot 2020-09-30 at 10.03.56 AM.png

 

You will need to do some work in Controller Services so check out the notes in the Red Box.  The flow is an example of how to generate many flow files and detect duplicates.  Be sure to do some research on the processor (google search) to understand how others have resolved working with the processor as your begin to integrate this into your own flow.    This community is also a great research tool too:

Screen Shot 2020-09-30 at 10.08.52 AM.png

 

 

 


 


If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  


 


Thanks,



Steven

Don't have an account?
Coming from Hortonworks? Activate your account here