Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to monitor NiFi cluster status?

How to monitor NiFi cluster status?

Contributor

I am wanting to know what the best way would be to monitor the status of a NiFi cluster. Currently I get Ambari alerts if the actual NiFi service stops, but I do not receive any alert if a NiFi node disconnects from the NiFi cluster. I was thinking of having something run within NiFi to get the total amount of nodes connected to the cluster and use this to send alerts. Is there some better way of doing this or generally accepted way?

4 REPLIES 4
Highlighted

Re: How to monitor NiFi cluster status?

Super Guru

@Josh Nicholson

You can use NiFi RestAPI to get cluster summary i.e how many nodes are connected, total node count..etc info

Curl Command:

curl -k  https://<nifi-host>:<port>/nifi-api/flow/cluster/summary
{"clusterSummary":{"connectedNodes":"n / n","connectedNodeCount":n,"totalNodeCount":n,"connectedToCluster":true,"clustered":true}}

As you we are going to get response as above json then you can parse the json using EvaluateJsonPath processor (or) ExtractText Processor to extract the values using regex then make a decision using RouteOnAttribute processor is totalNodeCount not equals to ConnectedNodeCount then trigger an email.

Flow:

1.InvokeHTTP
2.EvaluateJsonPath
3.RouteOnAttribute 4.PutEmail
Highlighted

Re: How to monitor NiFi cluster status?

Contributor

Thanks for the explanation @Shu. Do you know if there's any accepted way or best practice for monitoring the cluster status, or is this going to be the best way?

Highlighted

Re: How to monitor NiFi cluster status?

Super Guru
@Josh Nicholson

To monitor health/status of the cluster the REST API is the way to go, Alternative way is by using TailFile processor and tail nifi-app.log and send alerts if you find any disconnected node from the cluster.

this and this links walk you through configuring/usage of TailFile processor.

Refer to this link for more details regards to all kinds of monitoring in that NiFi offers.

Highlighted

Re: How to monitor NiFi cluster status?

Explorer

We already have a flow which monitors our cluster and sends user an email every time a node is disconnected. I am trying to create a process on NiFi where before it sends us the email saying the node is disconnected it should start or enable the disconnected node(s).

Don't have an account?
Coming from Hortonworks? Activate your account here