question Re: Nifi alert when a processor is in stopped state or running state in Support Questions

Nifi alert when a processor is in stopped state or running state

ep_gunner — Tue, 29 Mar 2022 14:03:00 GMT

Hello Team,

I have a pipeline that starts with an ad-hoc flow file processor as the starting processor. During cluster maintenance, the processors often go to a stopped state. Is there a way, I can get an alert with the note this processor is in a stopped state? Basically, the ask is to send alerts based on the state of the processor.

Thanks in advance.

Re: Nifi alert when a processor is in stopped state or running state

araujo — Wed, 30 Mar 2022 00:39:26 GMT

Hi @ep_gunner ,

You can use the QueryNiFiReportingTask reporting task to report on the status of any processors, connections, process groups, etc. This reporting task allow you to use a SQL query NiFi status.

For example, if you're interested in any stopped processor, you can use the query:

select * from PROCESSOR_STATUS where runStatus = 'Stopped'

You could also filter that by processor name or id if you need to be more specific:

select * from PROCESSOR_STATUS where runStatus = 'Stopped' and <YOUR_FILTER> where YOUR_FILTER could be any of the below: id = '<YOUR_PROCESSOR_ID>' groupId = '<YOUR_PROCESSOR_GROUP_ID>' name = '<YOUR_PROCESSOR_NAME>' processorType = '<YOUR_PROCESSOR_TYPE>' or any combination of the above

The results of this reporting task can be forwarded to any available ReportSink in NiFi (Database, Kafka, Email, etc). You can also send the results to a NiFi Input Port using the SiteToSiteReportingRecordSink and them you can process those reports with NiFi itself and take any other action you want.

Cheers,

André

Re: Nifi alert when a processor is in stopped state or running state

ep_gunner — Wed, 06 Apr 2022 12:49:07 GMT

Thanks for the reply @araujo

I was trying this solution out. The Nifi instance (1.16) I am using doesn't have QueryNifiReportingTask in the reporting task section, so I assume I have to build the nar first & then use it? Is that understanding correct?

Re: Nifi alert when a processor is in stopped state or running state

araujo — Fri, 08 Apr 2022 11:43:49 GMT

@ep_gunner ,

You can download the NAR from here: https://repo1.maven.org/maven2/org/apache/nifi/nifi-sql-reporting-nar/1.16.0/

Make sure you get the one that matches your NiFi version.

Cheers,

André

Re: Nifi alert when a processor is in stopped state or running state

rafy — Sat, 07 May 2022 09:04:42 GMT

If you do not mind, you can add MonitorActivity processor to the output end of the processor. This would generate two flow files(inactivity & activity restored) whenever your processor is down. In addition, connect the output end of MonitorActivity processor to PutEmail processor to configure an email Alert that would notify you when the processor is down & when activity has been restored.

I hope this helps.

Re: Nifi alert when a processor is in stopped state or running state

udayAle — Sat, 18 Mar 2023 15:59:05 GMT

Hello,

I am using nifi version 1.15, can you help me with the same problem.

Re: Nifi alert when a processor is in stopped state or running state

MattWho — Tue, 21 Mar 2023 18:40:08 GMT

@udayAle @ep_gunner
When NiFi is brought down, the current state (stopped, started, enabled, disabled) of all components is retained and on startup that same state is set on the components. Only time this is not true is when the property "nifi.flowcontroller.autoResumeState" is set to false in the nifi.properties file. When set to false a restart of NiFi would result in all components in a stopped state. In a production environment, this property should be set to true.

Perhaps you can share more details on the maintenance process you are using as I am not clear on how your maintenance is impacting the last known state of some components.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt