About mark_lin

DianaTorres · ‎09-27-2022

@arkumari As this is an older post, you would have a better chance of receiving a resolution by starting a new thread. This will also be an opportunity to provide details specific to your environment that could aid others in assisting you with a more accurate answer to your question. You can link this thread as a reference in your new post.

SAMSAL · ‎07-12-2022

Marfill, If backpressure is applied when the total number of flowfiles in a given queue has reached (#nodes * the limit per node) for example if you have a cluster of 3 nodes and the threshold is set to 10,000 then the backpressure will be applied when total # of flow files = 30,000 and so on. Regarding the Control Rate I believe its done per node statistics, for example if you have a control rate that allows 1 flow file per hour and the control rate processor is set part of load balancing on 3 nodes cluster, let say you receive total of 3 files for the first time one on each node then the 3 will be get processed immediately.

mark_lin · ‎05-08-2018

This seems to be a nifi 1.4/1.5 issue. This issue is reproducible in both nifi 1.4 and 1.5. However, it works well with nifi 1.6 arvo recordreader. Thanks, Mark

mark_lin · ‎03-23-2018

Hi Matt, The MonitorActivity processor is exceptionally useful. I will use it to monitor the overall health of my nifi processor groups. Thanks, Mark

MattWho · ‎03-21-2018

@Mark Lin Zookeeper is responsible for electing both the cluster coordinator and primary node for a nifi cluster. Reason why ZK may elect a new primary node include: Current primary node has not heartbeated to ZK: - possibly because network issues? - possibly because current Cluster Coordinator and/or Primary node is having issues preventing heartbeat form being sent such as Java garbage collection. As a stop-the-world event heartbeats would not be sent out while GC is running. by the time GC ends, ZK may have already elected a new Cluster coordinator and/or primary node. Node would notified of change next time it did successfully talk to ZK So you may never see node actually become disconnected from cluster. Node send heartbeats to current elected cluster coordinator. As long as those heartbeats are making it within configured timeouts, nodes will stay connected in cluster. Thanks you, Matt

mark_lin · ‎03-22-2018

Thanks Rahul. Using separate topic means good isolation. Directly using Nifi is a good option. Mark

mark_lin · ‎03-20-2018

Hi Matt, Thank you so much for the excellent, detailed explanation. Mark

Online	Offline
Last Visited	‎05-08-2018 04:08 PM

Member Since	‎02-09-2018 01:06 AM
Last Visited	‎05-08-2018 04:08 PM
Posts	18

Cloudera Community

Re: nifi cluster custom processor not work,nifi cu...

Re: nifi back pressure threshholds

Re: Nifi ConsumeKafakRecord_0_11 Throws Error "Cou...

Re: Dumy processor for error handling

Re: What could cause nifi primary/coordinator node...

Re: Share topics for different kafka connectors

Re: nifi repositories directory configurations