Created 11-02-2020 03:25 AM
Greetings,
I wanted to know how to set up a process where one will receive an email when ever a queue reaches its threshold limit , in apache nifi.
Thanks in advance.
Created 11-02-2020 05:15 AM
@P_Rat98 Per our PM discussion, in your flow use DetectDuplicate before sending an email. This should rate limit the # of messages you send based on your configuration of detectDuplicate. Additionally when this is linked in your flow, and duplicates are auto-terminated it will drain the flow and stop it from filling up the queue. Also as suggested you can chose to retain the duplicates, but move them into a much bigger Queue which isnt going to back up the main flow. Then once you see the email, you can go look at flow, see what flowfiles were causing issues, and take some corrective action.
If you really need to monitor flow for a queue being full, you would need to use the nifi API to check the status of the queue. This maybe more work than it is worth, when you can solve as above much easier. However, i would recommend you check the api out, there are a lot of api capabilities and I am beginning to use nifi api calls within my flow to monitor, stop, start, and take actions automatically that would normally require a human doing them in the UI.
If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.
Thanks,
Steven
Created 11-02-2020 05:15 AM
@P_Rat98 Per our PM discussion, in your flow use DetectDuplicate before sending an email. This should rate limit the # of messages you send based on your configuration of detectDuplicate. Additionally when this is linked in your flow, and duplicates are auto-terminated it will drain the flow and stop it from filling up the queue. Also as suggested you can chose to retain the duplicates, but move them into a much bigger Queue which isnt going to back up the main flow. Then once you see the email, you can go look at flow, see what flowfiles were causing issues, and take some corrective action.
If you really need to monitor flow for a queue being full, you would need to use the nifi API to check the status of the queue. This maybe more work than it is worth, when you can solve as above much easier. However, i would recommend you check the api out, there are a lot of api capabilities and I am beginning to use nifi api calls within my flow to monitor, stop, start, and take actions automatically that would normally require a human doing them in the UI.
If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.
Thanks,
Steven