NiFi email alerting with failure message which we see in bulletins ?

[#1]

I have a Process Group with 6 individual processor are running.So if one or two of Processors throws the error message in bulletins that needs to captured and sent an email with error message ?

Example workflow:

getfile---->Puts3object-->Updateattribute-->AttributeTOJSON-->ReplaceText-->PublishKafka

if workflow throws a error at Puts3object & AttributeTOJSON i need capture both bulletins message and then send the content to PutEmail processor .

Could someone give the idea how to achieve this ?

[#2]

@Veerendra Nath Jasthi In one of my NiFi Data Flows (multiple process groups) I create a Fail Output port in each group, and route that to a Notification Process Group's input port. which uses PutEmail Processor. If you are working in a single Process Group, just route all of your Failures to the same PutEmail Processor.

If this answer is helpful, please click ACCEPT.

[#3]

@Steven Matison

Could you please elaborate more ?

Right now I am using TailFile--->RouteOnContent-->PutEmail. So in this flow i am able to get the only errors from log file in attached file in an email.

My question is how can I capture those error message and put as the message of PutEmail processor ?

[#4]

@Veerendra Nath Jasthi

In PutEmail do you have AttachFile true or false? That attribute is underneath the scroll in the processor modal window so maybe you missed that setting?

If you are trying to get the content of the flowfile into the message body, without an attachment, you will need to use the ExtractText processor to get the text from the content of a flow file into an attribute. You should be able to add a property like "message.body" with a value of "(.*)" to ExtractText to get it to extract everything. Then use the PutEmail Attribute Include All Attributes In Message set to True.

[#5]

@Steven Matison

Thanks for your quick response and It is working fine now but I have two concerns here:

1.I am getting bunch of emails with same error message rather I would only need one email with all error message.

2.If I include all error message in message body it looks so weird so I am planning to message body contains only "Process Group Name" & "Process Group ID". For process group name I found in HWX blog that ExcuteScript processor will do with below code:

1. def flowFile = session.get()
2. if(!flowFile)return
3. processGroupName = context.procNode?.getProcessGroup().getName()
4. flowFile = session.putAttribute(flowFile,'processGroupName', processGroupName)
5. session.transfer(flowFile, REL_SUCCESS)

Where should I have ExcuteScript processor in my current flow ?

(my current flow is :TailFile--->RouteOnContent-->ExtractText-->PutEmail.)

How would I configure that my message body contains only "Process Group Name" & "Process Group ID" ?

Thanks,

Jasthi.

[#6]

@Veerendra Nath Jasthi

You probably want it before PutEmail. Your ExecuteScript should be able to set flowfile content or attributes based on the configuration of the script. If you dont need to pass other attributes into putEmail, i would write the ProcessGroupName, Process ID, and Timestamp to flowfile content. Then use MergeContent to join multiple instances of the errors into a single flow file, and pass this to PutEmail. Hope this makes sense.

I would also recommend trying to resolve the errors. A good NiFi Data Flow should not have any errors or should be fault tolerate with a method to route the failures back into the flow.

Search HCC for "executescript cookbook" to find a great 3 part article on ExecuteScript.

[#7]

Hi Veerendra,

it would be great if you could share some screenshots of the solution. It will be helpful for the users to learn new scenarios.

Thanks!!!

[#8]

@Sharath V Yeah sure sharath. Here I have attached the screen shot of my current workflow which is running in the production to get email whenever any processor fails it will alert you with email and the reason.nifi-email-alert.png

In attached flow I was tailing my nifi-app.log file with current time and then routing the content if it contains with ERROR. Then I am executing the script for getting the process group name wherever particular processor failed below is the script :

def flowFile = session.get() if(!flowFile) return processGroupName = context.procNode?.getProcessGroup().getName() processGroupId = context.procNode?.processGroupIdentifier ?: 'unknown' flowFile = session.putAttribute(flowFile, 'processGroupName', processGroupName) flowFile = session.putAttribute(flowFile, 'processGroupId', processGroupId) session.transfer(flowFile, REL_SUCCESS)

And finally I am sending both error message and process group name in the email with attached file.

[#9]

@Jasthi  How can you get the processgroupname and ID ? this script get the current processgroup and not the one where the error occur .

I am facing the same problem and i trying to create a log error flow that logs everything on the server , right now i was able to get the error and send by email , but there is nothing that indentifies in what flow the error happened