Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to notify user if there is any failure/error in nifi from bulletin board?

Solved Go to solution
Highlighted

How to notify user if there is any failure/error in nifi from bulletin board?

Team,

I am new to Nifi Application. Can anyone please let me know how to notify user if there is any failure/error in Nifi Processors. I know error message is captured in Bulletin Board. Is it possible to send email to user if there is any error in bulletin board? Any help is much appreciated.

Thanks

Gowthaman

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: How to notify user if there is any failure/error in nifi from bulletin board?

Master Guru

@Gowthaman Jayachandran

The NiFi master bulletin board does not constantly run in the background, Its simply shows a continuous stream of bulletins being produced by processor components on the canvas while it is open. The bulletins are nothing more then snippets of the WARN or ERROR log lines that are being written to the nifi-app.log.

The processors that are actually producing bulletins often have a a failure relationship where such FlowFiles will be routed upon failure. This is where you want to handle your notifications. Perhaps setting up a failure loop that will attempt to processes the FlowFile x number of times before kicking it out of the loop. If a file gets kicked out of the loop, you could use and putEmail processor to notify you of the failure.

See below for template of such a failure loop:

https://cwiki.apache.org/confluence/download/attachments/57904847/Retry_Count_Loop.xml?version=1&mod...

For processor components where there is no failure relationship, you could use a MonitorActivity processor in your dataflow. This processor can be configured with a threshold and if a FlowFile has not passed within that threshold, a FlowFile is generated that can also be send to a PutEmail processor for alerting purposes. It also will Generate a FlowFile when activity has been restored.

Thanks,

Matt

View solution in original post

2 REPLIES 2
Highlighted

Re: How to notify user if there is any failure/error in nifi from bulletin board?

Master Guru

@Gowthaman Jayachandran

The NiFi master bulletin board does not constantly run in the background, Its simply shows a continuous stream of bulletins being produced by processor components on the canvas while it is open. The bulletins are nothing more then snippets of the WARN or ERROR log lines that are being written to the nifi-app.log.

The processors that are actually producing bulletins often have a a failure relationship where such FlowFiles will be routed upon failure. This is where you want to handle your notifications. Perhaps setting up a failure loop that will attempt to processes the FlowFile x number of times before kicking it out of the loop. If a file gets kicked out of the loop, you could use and putEmail processor to notify you of the failure.

See below for template of such a failure loop:

https://cwiki.apache.org/confluence/download/attachments/57904847/Retry_Count_Loop.xml?version=1&mod...

For processor components where there is no failure relationship, you could use a MonitorActivity processor in your dataflow. This processor can be configured with a threshold and if a FlowFile has not passed within that threshold, a FlowFile is generated that can also be send to a PutEmail processor for alerting purposes. It also will Generate a FlowFile when activity has been restored.

Thanks,

Matt

View solution in original post

Re: How to notify user if there is any failure/error in nifi from bulletin board?

Master Guru
@Gowthaman Jayachandran

You can also retrieve bulletins via a call to NiFi's rest-api:

curl 'http://<hostname>:<port>/nifi-api/flow/bulletin-board'

Don't have an account?
Coming from Hortonworks? Activate your account here