Created on 05-05-201707:33 PM - edited 08-17-201901:08 PM
Recently I was asked how to Monitor and alert while MonitorDiskUsage or MonitorMemory Reporting Task WARNINGS are generated when it exceeds a predefined threshold, I am trying to capture the steps to implement the same.
To test this, Make sure HDF-2.x version of NiFi is up an running.
You Already have a MonitorDiskUsage/MonitorMemory Reporting task created and configured, set a lower threshold to force generate the alert while testing.
3. Make a note of the Reporting Task uuid to be monitored:
Creating a Flow to Monitor Reporting Task.
Drop a GenerateFlowFile to execute every 5mins, with 1 byte file to trigger the flow.
2. Drop an UpdateAttribute processor to the canvas with below configuration:
NIFI_HOST : <Your NiFi FQDN>
NIFI_PORT : <Your NiFi http port>
REPORTING-TASK-UUID : <Your MonitorDiskUsage or MonitorMemory controller service uuid>
3. Connect Success relationship of GenerateFlowFile to UpdateAttribute.
4. Drop a InvokeHTTP processor to the canvas, and configure it as below:
5. Connect Success relation of UpdateAttribute to InvokeHTTP and auto terminate all relationships of InvokeHTTP processor but Response relationship.
6. Drop a EvaluateJsonPath processor to the canvas with below configuration:
7. Auto terminate All relationship of EvaluateJsonPath processor except Matched relationship and connect InvokeHTTP processor’s Response relation to it.
8. Drop a SplitJson processor to canvas with below Configurtaion. [The reason for splitting json is because the REST call to reporting task gives duplicate json array in the result, which contains details we need.]
9. Connect Matched relationship of EvaluateJsonPath to SplitJson processor.
10. Drop another EvaluateJsonPath processor to the canvas with below configuration:
15. Auto terminate all relationships of PutEmail processor and connect Success relationship of ControlRate processor to it.
16. Now you have your flow ready and can start it to monitor and sent Email Alert for UI Notification thrown by NiFi when Disk or Memory utilization exceeds given threshold. The Flow Would Look like below:
Staring the flow, Reporting task and testing it
1. Now lets start the MonitorDiskUsage Reporting task to generate an Alert to test it with below configuration with a lower threshold to force generate the alert. I am monitoring disk on my mac and it’s disk is more than 50% utilized, so I will get this WARNING in the NiFi UI.
2. As soon as this Warning show up and the flow created is running you will get alert in your inbox stating the same, similar as below:
3. This concludes the tutorial for Monitoring your Reporting Tasks with NiFi itself.
4. Too lazy to create the flow???.. Download my template here