Member since
07-30-2019
3406
Posts
1623
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 320 | 12-17-2025 05:55 AM | |
| 381 | 12-15-2025 01:29 PM | |
| 366 | 12-15-2025 06:50 AM | |
| 358 | 12-05-2025 08:25 AM | |
| 599 | 12-03-2025 10:21 AM |
03-23-2018
06:51 PM
Hi Matt, The MonitorActivity processor is exceptionally useful. I will use it to monitor the overall health of my nifi processor groups. Thanks, Mark
... View more
04-01-2018
04:23 PM
1 Kudo
@ANKIT PATEL Did the answer help in the resolution of your query? Please close the thread by marking the answer as Accepted!
... View more
03-21-2018
06:47 PM
@Mark Lin Zookeeper is responsible for electing both the cluster coordinator and primary node for a nifi cluster. Reason why ZK may elect a new primary node include: Current primary node has not heartbeated to ZK: - possibly because network issues? - possibly because current Cluster Coordinator and/or Primary node is having issues preventing heartbeat form being sent such as Java garbage collection. As a stop-the-world event heartbeats would not be sent out while GC is running. by the time GC ends, ZK may have already elected a new Cluster coordinator and/or primary node. Node would notified of change next time it did successfully talk to ZK So you may never see node actually become disconnected from cluster. Node send heartbeats to current elected cluster coordinator. As long as those heartbeats are making it within configured timeouts, nodes will stay connected in cluster. Thanks you, Matt
... View more
03-27-2018
12:33 PM
@Jayendra Patil You currently have 120 set as your "maximum Timer Driven Thread Count". Multiply that by the number of nodes in your NiFi cluster to see maximum number of useable threads cumulative across you cluster. Then look at the info bar across the top of your canvas. Does it look like your dataflows is using all these threads you have allocated? You may need to make adjustments to your processor configurations to maximize the thread usage. Look for where you have bottlenecks in your dataflow (queues built up in front of processors). What kind of processors reading from these built up queues? How have they been configured? Just because you allocated more available threads does not mean NiFi processors are going to automatically start using them or even be allowed to use them.
... View more
10-02-2018
01:04 PM
@Arun A K The "ldap-user-group-provider" is not passed any input, so using things like (cn={0}) will not work. This is perfectly valid in the login-identity-providers.xml since it is being passed the login name the user enters on login screen. <userGroupProvider>
<identifier>ldap-user-group-provider</identifier> <class>org.apache.nifi.ldap.tenants.LdapUserGroupProvider</class> <property name="Authentication Strategy">SIMPLE</property> <property name="Manager DN">uid=admin,cn=blah,cn=blah,dc=blah,dc=com</property> <property name="Manager Password">blah</property> <property name="TLS - Keystore"></property> <property name="TLS - Keystore Password"></property> <property name="TLS - Keystore Type"></property> <property name="TLS - Truststore"></property> <property name="TLS - Truststore Password"></property> <property name="TLS - Truststore Type"></property> <property name="TLS - Client Auth"></property> <property name="TLS - Protocol"></property> <property name="TLS - Shutdown Gracefully"></property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url">ldap://blah.ldap.com:389</property> <property name="Page Size"></property> <property name="Sync Interval">30 mins</property> <property name="User Search Base">cn=users,cn=accounts,dc=blah,dc=blah,dc=com</property> <property name="User Object Class">person</property> <property name="User Search Scope">SUBTREE</property> <property name="User Search Filter"></property> <property name="User Identity Attribute">uid</property> <property name="User Group Name Attribute">memberOf</property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="Group Search Base">cn=groups,cn=accounts,dc=blah,dc=blah,dc=com</property> <property name="Group Object Class">posixgroup</property> <property name="Group Search Scope">SUBTREE</property> <property name="Group Search Filter"></property> <property name="Group Name Attribute">cn</property> <property name="Group Member Attribute">member</property> <property name="Group Member Attribute - Referenced User Attribute"></property>
</userGroupProvider> The above is what you should have at a minimum. You may need to also specify the two "Referenced User/Group Attribute" values, but can't tell from your sample LDAP output is that is really going to be necessary. - Thanks, Matt
... View more
03-07-2018
02:04 PM
Hello Matt, Sorry for the delay, but I did not see your answer before. I needed to support server and client Authentification. I ended up using a different keystore for storing the various key I used. Best regards Abdou
... View more
02-20-2018
08:11 PM
1 Kudo
I stand corrected. It seems if you are doing Tailfile in a cluster, the directories and files must exist on all hosts.
... View more
05-07-2018
04:22 PM
@Raffaele S NiFi is a very difficult things to make a one size fits all sizing recommendation for. NiFi does not typically scale linearly. This is why you see the hardware specs exponentially increase as throughput increases. This is based on the fact that typical NiFi workflows all grow exponentially in size and complexity as the volume of throughput increases in most cases. More and more workflows are added. - Different NiFi processors in different workflows contribute to different server resource usage. That resource usage varies based processor configuration and FlowFile volume. So even two workflows using same processors may have different sizing needs. - How well a NiFi is going to perform has a lot to do with the workflow the user has build. After all it is this user designed work flow that is going to be using the majority of the resources on each node. - Best answer to be honest is to built your workflows and stress test them. This kind of a modeling and simulation setup. Learn the boundaries your workflows put on your hardware. At what data volume point does CPU utilization, network bandwidth, memory load, disk IO become my bottleneck for my specific workflow(s). Tweaking your workflows and component configurations. Then scale out by adding more nodes allowing some headroom considering it is very unlikely ever node will be processing the exact same number of NiFi FlowFiles all the time. - Thanks, Matt
... View more
01-29-2018
04:58 PM
@balalaika By Default HDF install the NiFi service to run as the "nifi" user. You issue is likely that the nifi user is unable to navigate down that directory tree to the target script. Try becoming the nifi user (su - nifi) and navigating to the script directory. Thank you, Matt If this answer has addressed your question, please take a moment to click the "accept" link below the answer.
... View more