Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Sending an email whenever the Queue is Full- Nifi

avatar
Explorer

Greetings, 

 I wanted to know how to set up a process where one will receive an email when ever a queue reaches its threshold limit , in apache nifi.

Thanks in advance.

 

 

 

 

1 ACCEPTED SOLUTION

avatar
Super Guru

@P_Rat98   Per our PM discussion,  in your flow use DetectDuplicate before sending an email.   This should rate limit the # of messages you send based on your configuration of detectDuplicate.  Additionally when this is linked in your flow, and duplicates are auto-terminated it will drain the flow and stop it from filling up the queue.    Also as suggested you can chose to retain the duplicates, but move them into a much bigger Queue which isnt going to back up the main flow.  Then once you see the email, you can go look at flow, see what flowfiles were causing issues, and take some corrective action.

 

If you really need to monitor flow for a queue being full,  you would need to use the nifi API to check the status of the queue.  This maybe more work than it is worth, when you can solve as above much easier.  However, i would recommend you check the api out,  there are a lot of api capabilities and I am beginning to use nifi api calls within my flow to monitor, stop, start, and take actions automatically that would normally require a human doing them in the UI.

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

 

Thanks,

Steven

View solution in original post

1 REPLY 1

avatar
Super Guru

@P_Rat98   Per our PM discussion,  in your flow use DetectDuplicate before sending an email.   This should rate limit the # of messages you send based on your configuration of detectDuplicate.  Additionally when this is linked in your flow, and duplicates are auto-terminated it will drain the flow and stop it from filling up the queue.    Also as suggested you can chose to retain the duplicates, but move them into a much bigger Queue which isnt going to back up the main flow.  Then once you see the email, you can go look at flow, see what flowfiles were causing issues, and take some corrective action.

 

If you really need to monitor flow for a queue being full,  you would need to use the nifi API to check the status of the queue.  This maybe more work than it is worth, when you can solve as above much easier.  However, i would recommend you check the api out,  there are a lot of api capabilities and I am beginning to use nifi api calls within my flow to monitor, stop, start, and take actions automatically that would normally require a human doing them in the UI.

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

 

Thanks,

Steven