Hello, guys! I still have a three-node cluster Apache NiFi 1.18.0 (yeap, upgrade task in by backlog =)) and trying to setup it behind HAProxy just for UI (/nifi endpoint). I wanna to implement a health check to balance user requests only to nodes in CONNECTED status to avoid issues when nodes moves to maintenance (offload, disconnect). Is it good idea, right? Or maybe I do some overengineering things? 😃 By the way, I have next HAProxy config file: frontend nifi_443 bind *:443 ssl crt /etc/haproxy/ssl/cert_file.pem http-request set-path /nifi if { path / } mode http option httplog acl DST_IP dst 10.0.0.10 use_backend nifi_ui_9443 if DST_IP backend nifi_ui_9443 mode http option httpchk http-check send meth GET uri /nifi-api/access/config ver HTTP/1.1 hdr Host nifi-cluster.corp.company.com http-check expect status 200 cookie SERVERID insert indirect nocache balance leastconn server srv_1 10.0.0.1:9443 check ssl verify none cookie web1 server srv_2 10.0.0.2:9443 check ssl verify none cookie web2 server srv_3 10.0.0.3:9443 check ssl verify none cookie web3 Basically it works fine when all nodes are CONNECTED, but if some node goes to DISCONNECTED state, http-check to selected API method still returns 200 and HAProxy continues to route users to that node. So, give an advice, please. How can I configure HAProxy health-check to get expected behavior? Thanks a lot. ... View more

Hi, everyone. We have a three-node Apache NiFi Cluster 1.18.0. Content archive is enabled (15 minutes deep), content repository is on it's own disk. Today we noticed that 2nd node use a lot of disk space by content repository (>80%). I've checked archive files and they fill only about 500 Mb disk space. Other space was used by current content data. Other nodes used content repo as usual -- about 25%. Because we start to receive next errors and 2nd node performance goes down, I've start 2nd node offloading (~ 13:10 on graph down below): Unable to write flowfile content to content repository container repo0 due to archive file size constraints; waiting for archive cleanup. Total number of files currently archived = 63 After 2nd node was successfully offloaded I've connect it to the cluster again and now it works fine -- data processing, content repo uses as usual. But when offloading processed, I've track disk space utilization graphs and noticed that 1st and 3rd node did not utilize disk space proportional that 2nd node frees: As you can see on screenshot, 2nd node free about 160 Gb of disk space, but 1st got only about +15 Gb and 3rd -- +6 Gb. Did we loose data? Or other nodes just process in place all data from 2nd while offloading process?   ... View more

Hello, everyone. I have an issue with NiFi Registry 1.20 -- UI button "Add user" is inactive with Initial Admins account. But, if I check the user permissions, it looks good enough: The Initial Admin user is LDAP user and also member of "nifi admins" LDAP group. My authorizers.xml file: <?xml version="1.0" encoding="UTF-8" standalone="yes"?> <authorizers> <userGroupProvider> <identifier>ldap-user-group-provider-1</identifier> <class>org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider</class> <property name="Authentication Strategy">LDAPS</property> <property name="Manager DN">user</property> <property name="Manager Password">password</property> <property name="TLS - Keystore">./conf/keystore.jks</property> <property name="TLS - Keystore Password">pass</property> <property name="TLS - Keystore Type">JKS</property> <property name="TLS - Truststore">./conf/truststore.jks</property> <property name="TLS - Truststore Password">pass</property> <property name="TLS - Truststore Type">JKS</property> <property name="TLS - Client Auth">None</property> <property name="TLS - Protocol">TLSv1.2</property> <property name="TLS - Shutdown Gracefully"></property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">60 secs</property> <property name="Url">ldaps://ldap.company.com</property> <property name="Page Size">500</property> <property name="Sync Interval">5 mins</property> <property name="Group Membership - Enforce Case Sensitivity">false</property> <property name="User Search Base">ou</property> <property name="User Object Class">user</property> <property name="User Search Scope">SUBTREE</property> <property name="User Search Filter">some filter</property> <property name="User Identity Attribute">sAMAccountName</property> <property name="User Group Name Attribute"></property> <property name="User Group Name Attribute - Referenced Group Attribute"></property> <property name="Group Search Base">ou</property> <property name="Group Object Class">group</property> <property name="Group Search Scope">SUBTREE</property> <property name="Group Search Filter">some search filter</property> <property name="Group Name Attribute">cn</property> <property name="Group Member Attribute">member</property> <property name="Group Member Attribute - Referenced User Attribute">distinguishedName</property> </userGroupProvider> <userGroupProvider> <identifier>ldap-user-group-provider-2</identifier> .... </userGroupProvider> <userGroupProvider> <identifier>ldap-user-group-provider-3</identifier> .... </userGroupProvider> <userGroupProvider> <identifier>ldap-user-group-provider-4</identifier> .... </userGroupProvider> <userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class> <property name="Users File">/opt/nifi-registry/nifi-registry-current/conf/authorizations/users.xml</property> <property name="Initial User Identity 1">cn=some_init_user_for_nifi_01, ou=nifi</property> <property name="Initial User Identity 2">cn=some_init_user_for_nifi_02, ou=nifi</property> <property name="Initial User Identity 3">cn=some_init_user_for_nifi_03, ou=nifi</property> </userGroupProvider> <userGroupProvider> <identifier>composite-configurable-user-group-provider</identifier> <class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class> <property name="User Group Provider 0">file-user-group-provider</property> <property name="User Group Provider 1">ldap-user-group-provider-1</property> <property name="User Group Provider 2">ldap-user-group-provider-2</property> <property name="User Group Provider 3">ldap-user-group-provider-3</property> <property name="User Group Provider 4">ldap-user-group-provider-4</property> </userGroupProvider> <accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class> <property name="User Group Provider">composite-configurable-user-group-provider</property> <property name="Authorizations File">/opt/nifi-registry/nifi-registry-current/conf/authorizations/authorizations.xml</property> <property name="Initial Admin Identity">init_admin_name</property> </accessPolicyProvider> <authorizer> <identifier>managed-authorizer</identifier> <class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class> <property name="Access Policy Provider">file-access-policy-provider</property> </authorizer> </authorizers> Could you guess please what is wrong? ... View more