I dont have processor Nifi Site To Site Status Reporting Task in Nifi 1.11.4

[#1]

Hello, Greetings to the entire community, I have a problem, we currently have a nifi cluster with version 1.11.4, I need to send alerts to datadog, and the SiteToSiteStatusReportingTask processor, it helps me obtain error information in the flows that we have developed as I understand, however this processor does not appear for the version of Nifi that we have, is there any other way to obtain the information of the flows with errors or any maintenance of updating or putting a patch nifi-site-to-site-reporting-nar-1.11.4.nar

I would appreciate any guidance or support.

[#2 · Accepted solution]

@Bern 

I am having difficulty clearly understanding your question.

I will start by saying that Apache NIFi 1.11.4 was released way back in 2019 and will have many unresolved CVEs.  I strongly encourage you to at least upgrade to the latest available NiFi 1.x release 1.28.1 as I know migrating to Apache NiFi 2.x versions takes a good amount of planning and likely some dataflow redesign work.

I think first we need to get our terminology correct, so we can communicate clearly on the issue/question.

NiFi processors are what you add to the canvas that perform specific tasks.  Processors will have connections that allows you to connect a processor with another component (processor, input port, output port, funnel, etc).   NiFi Reporting Tasks are added via the NiFi controller and perform their function in the background.  Then you also have NiFi Controller Services which are services that are used by other components (processors for example).

The SiteToSiteStatusReportingTask NiFi reporting task has been a part of Apache NiFi since 1.2.0 release, so it does exist in your 1.1.4 version.  

The screenshot you shared is showing a bunch of Controller Services, so you are in the wrong UI for adding a Reporting Task.

You can find and add NiFi Reporting task by clicking on the NIFi Global menu in the upper right corner of the UI and selecting "Controller Settings" from the displayed menu:

From the UI that appears, you will be able to select the "Reporting Tasks" tab:

Click the box to the far right with the "+" symbol to bring up the UI for selecting the Reporting task you wish to add.

NOTE: The list of available Reporting Tasks will vary by Apache NIFi release version.

What I actually think you will want to use is the SiteToSiteBulletinReportingTask reporting task. You can use this Reporting task to send bulletins that your processors are producing to a NiFi remote Input Port.  Your processors generate ERROR bulletins by default when issues occur, so you can build a dataflow that will process these bulletins send to it via this reporting task and do alerting as you need. For example: send an email using the putEmail processor to alert someone about specific errors. 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#3 · Accepted solution]

@Bern 

I am having difficulty clearly understanding your question.

I will start by saying that Apache NIFi 1.11.4 was released way back in 2019 and will have many unresolved CVEs.  I strongly encourage you to at least upgrade to the latest available NiFi 1.x release 1.28.1 as I know migrating to Apache NiFi 2.x versions takes a good amount of planning and likely some dataflow redesign work.

I think first we need to get our terminology correct, so we can communicate clearly on the issue/question.

NiFi processors are what you add to the canvas that perform specific tasks.  Processors will have connections that allows you to connect a processor with another component (processor, input port, output port, funnel, etc).   NiFi Reporting Tasks are added via the NiFi controller and perform their function in the background.  Then you also have NiFi Controller Services which are services that are used by other components (processors for example).

The SiteToSiteStatusReportingTask NiFi reporting task has been a part of Apache NiFi since 1.2.0 release, so it does exist in your 1.1.4 version.  

The screenshot you shared is showing a bunch of Controller Services, so you are in the wrong UI for adding a Reporting Task.

You can find and add NiFi Reporting task by clicking on the NIFi Global menu in the upper right corner of the UI and selecting "Controller Settings" from the displayed menu:

From the UI that appears, you will be able to select the "Reporting Tasks" tab:

Click the box to the far right with the "+" symbol to bring up the UI for selecting the Reporting task you wish to add.

NOTE: The list of available Reporting Tasks will vary by Apache NIFi release version.

What I actually think you will want to use is the SiteToSiteBulletinReportingTask reporting task. You can use this Reporting task to send bulletins that your processors are producing to a NiFi remote Input Port.  Your processors generate ERROR bulletins by default when issues occur, so you can build a dataflow that will process these bulletins send to it via this reporting task and do alerting as you need. For example: send an email using the putEmail processor to alert someone about specific errors. 

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#4]

Hi MattWho

I'm surprised by the speed. Thank you very much for the prompt response. The correction you made regarding the technical terms is indeed completely valid. I made a mistake when referring to a controller as a processor, but you managed to understand me anyway. I tried to implement it from the context menu in the canvas root, and indeed, it didn't appear. However, I followed the instructions to access it from the global menu and was indeed able to do so, so I'm very grateful. Now, regarding the NIFI version, I opened a post a while back when we tried to migrate and it didn't work on an AWS EC2 instance currently running the NIFI 1.11.4 cluster. We tried versions 1.16.3 and 1.28.1, and neither launched the canvas. However, the status indicated it was running but didn't record anything in the log as a failure. They weren't able to provide further support at the time, so we gave up on the migration.

[#5]

Thanks.

[#6 · Accepted solution]

@Bern 

For NiFi site-to-site (S2S), you can NOT have each node configured differently (other then each node's unique  hostname being set).  The way Site-To-Site works is as follows:

• The Destination URL is configured with a comma separated list of NiFi URLs for the hosts in the target NiFi cluster (adding a comma separated list allows S2S to still function if one of the nodes in the target cluster is down).  So you can configure just one target URL if you want and it will still work.   If you your NiFi cluster is secured, the the destination URLS must also be https urls.
• So S2S attempts to connect to the first URL in the list to fetch S2S details (number of nodes in cluster, cluster hostnames, is http enabled, RAW port of each node, load on each node, etc) about the target cluster.  The S2S details are rechecked every 30 seconds to see if they have changed (for example adding another node or removing a node from target cluster). 
• Then S2S uses that information to distribute FlowFile across all nodes in the destination NiFi cluster.  The client (SitetoSiteStatusReprotingTask) dictates whether you want to use RAW or HTTP transport protocols.  If  using RAW, make sure the RAW port is not in use on any of the nodes already.
  
  

Take a look in the nifi-app.log for the exception as it is likely to include a full stack trace with it that may shed more light on your issue.

It would be hard for me to say exactly what you issue is unless i knew your NiFi setup (nifi.properties) and the specific configuration of your SiteToSiteStatusReporting Task. What do you encounter if you use HTTP instead of RAW transport protocol?

I'd also suggest starting a new Community question as this new question is not related to your original question in this post.

Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt

[#7]

I opened a new message because the initial context of this thread already supported me with its solution and this would be another query where I attached the properties and the flow.   

https://community.cloudera.com/t5/Support-Questions/Site-to-Site-Status-Reporting-Task-Error-Notific...