Nifi alert when a processor is in stopped state or running state

[#1]

Hello Team,

I have a pipeline that starts with an ad-hoc flow file processor as the starting processor. During cluster maintenance, the processors often go to a stopped state. Is there a way, I can get an alert with the note this processor is in a stopped state? Basically, the ask is to send alerts based on the state of the processor.

Thanks in advance.

[#2]

Hi @ep_gunner ,

You can use the QueryNiFiReportingTask reporting task to report on the status of any processors, connections, process groups, etc. This reporting task allow you to use a SQL query NiFi status.

For example, if you're interested in any stopped processor, you can use the query:

select *
from PROCESSOR_STATUS
where runStatus = 'Stopped'

You could also filter that by processor name or id if you need to be more specific:

select *
from PROCESSOR_STATUS
where runStatus = 'Stopped'
  and <YOUR_FILTER>

where YOUR_FILTER could be any of the below:
  id = '<YOUR_PROCESSOR_ID>'
  groupId = '<YOUR_PROCESSOR_GROUP_ID>'
  name = '<YOUR_PROCESSOR_NAME>'
  processorType = '<YOUR_PROCESSOR_TYPE>'

or any combination of the above

The results of this reporting task can be forwarded to any available ReportSink in NiFi (Database, Kafka, Email, etc). You can also send the results to a NiFi Input Port using the SiteToSiteReportingRecordSink and them you can process those reports with NiFi itself and take any other action you want.

Cheers,

André

[#3]

Thanks for the reply @araujo 

I was trying this solution out. The Nifi instance (1.16) I am using doesn't have QueryNifiReportingTask in the reporting task section, so I assume I have to build the nar first & then use it? Is that understanding correct?

[#4]

@ep_gunner ,

You can download the NAR from here: https://repo1.maven.org/maven2/org/apache/nifi/nifi-sql-reporting-nar/1.16.0/

Make sure you get the one that matches your NiFi version.

Cheers,

André

[#5]

If you do not mind, you can add MonitorActivity processor to the output end of the processor. This would generate two flow files(inactivity & activity restored) whenever your processor is down. In addition, connect the output end of MonitorActivity processor to PutEmail processor to configure an email Alert that would notify you when the processor is down & when activity has been restored.

I hope this helps.

[#6]

Hello,

I am using nifi version 1.15, can you help me with the same problem.

[#7]

@udayAle @ep_gunner 
When NiFi is brought down, the current state (stopped, started, enabled, disabled) of all components is retained and on startup that same state is set on the components.  Only time this is not true is when the property "nifi.flowcontroller.autoResumeState" is set to false in the nifi.properties file.  When set to false a restart of NiFi would result in all components in a stopped state.  In a production environment, this property should be set to true. 

Perhaps you can share more details on the maintenance process you are using as I am not clear on how your maintenance is impacting the last known state of some components.

If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped.

Thank you,

Matt