@sha257  In your ldap-identity-provider located in the login-providers.xml file: <provider> <identifier>ldap-identity-provider</identifier> <class>org.apache.nifi.registry.security.ldap.LdapIdentityProvider</class> <property name="Authentication Strategy">LDAPS</property> <property name="Manager DN">CN=ABC1234,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Manager Password">xxxx</property> <property name="TLS - Keystore">/opt/nifi-registry/nifi-registry-current/conf/keystore.jks</property> <property name="TLS - Keystore Password">xxx</property> <property name="TLS - Keystore Type">jks</property> <property name="TLS - Truststore">/opt/nifi-registry/nifi-registry-current/conf/truststore.jks</property> <property name="TLS - Truststore Password">xxxx</property> <property name="TLS - Truststore Type">JKS</property> <property name="TLS - Client Auth">NONE</property> <property name="TLS - Protocol">TLS</property> <property name="TLS - Shutdown Gracefully">false</property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url">ldaps://example-v3-prd.corp1.ad1.xyz.net:636</property> <property name="User Search Base">OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="User Search Filter">(cn={0})</property> <property name="Identity Strategy">USE_DN</property> <property name="Authentication Expiration">12 hours</property> </provider> I would change "Identity Strategy" from "USE_DN" to "USE_USERNAME".   At the NiFi login you are logging in with your ldap CN user string.  That user string gets substituted in place of the "{0}" in the "User Search Filter.   So you'll want that same string entered at login to be your user identity.  This will allow your user to map correctly with the user identity returned by your ldap-user-group-provider. In the file-access-policy-provider you'll want to set the "Initial Admin Identity" to the username you login with rather then the full DN. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@sha257  There is a lot shared here and I see numerous configuration issues and have some questions. ---->  authorizers.xml  <---- It is easiest to read the authroizers.xml from bottom up (the bottom of the config file will have the authorizer: <authorizer> <identifier>managed-authorizer</identifier> <class>org.apache.nifi.registry.security.authorization.StandardManagedAuthorizer</class> <property name="Access Policy Provider">file-access-policy-provider</property> </authorizer> </authorizers> You can see the authorizer above is loading policies from the "file-access-policy provider", so we scroll up to find that: <accessPolicyProvider> <identifier>file-access-policy-provider</identifier> <class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class> <property name="User Group Provider">file-user-group-provider</property> <property name="Authorizations File">./conf/authorizations.xml</property> <property name="Initial Admin Identity">CN=ABC1234,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Nifi Identity 1">CN=ABC1234,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Nifi Identity">CN=ABC1234,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="NiFi Group Name"/> <!--<property name="NiFi Identity 1"></property>--> </accessPolicyProvider> *** This provider will create the authorizations.xml file ONLY if it does not already exist. *** Here we see you have configured your "Initial Admin Identity" with the full DN for your user.  This full DN string will be used for setting up the required authorizations for this user Identity to be able to access NiFi and perform administrative tasks (setup new user authorizations, remove access, setup admin level configurations, etc.). Keep in mind that user identities are case sensitive in NiFi.   We can also see what provider is being used to provide the current list of user and group identities upon which policies can be set. Here we see it is configured to use "file-user-group-provider". Question: Why do you have the same NiFi host certificate DN configured twice in NiFi Identity and NiFi Identity 1? Now we scroll up to the file-user-group-provider: <userGroupProvider> <identifier>file-user-group-provider</identifier> <class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class> <property name="Users File">./conf/users.xml</property> <property name="Initial User Identity 1">CN=ABC1234,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Initial User Identity admin">cn=CEF567,ou=Solid Users,ou=User Accounts,dc=corp1,dc=ad1,dc=xyz,dc=net</property> </userGroupProvider> *** This provider will create the users.xml file ONLY if it does not already exist. *** This is one of several providers that can provide initial user identity strings and group identity strings to the configured policy provider for setting up access policies.  Here we also see that you used the DN for your "initial user identity admin".   While your authorizers.xml has a composite-user-group-provider and ldap-user-group-provider also configured, they are not being used since there is not configuration path to them from the authorizer. Question:  Does your LDAP contain entries for your NiFi hosts (not common)?  If it does not you will still need the file-user-group-provider to provide those host user identities. Let's assume you need both "file-user-group-provider" and "ldap-user-group-provider". Your authorizers.xml would need to have this structure: Managed-authorizer (configured to use file-access-policy-provider) ---> file-access-policy-provider (configured to use composite-configurable-user-group-provider) ---> composite-configurable-user-group-provider (configured to use both configurable provider: file-user-group-provider and user group provider: ldap-user-group-provider).   Then you need both file-user-group-provider and ldap-user-group-provider configured. *** Note: You can NOT have multiple user group providers return the same user-identity or group identity strings.  So you'll need to remove the initial admin user from the file-user-group-provider if that admin identity is now going to be returned by the ldap-user-group-provider. Now when it comes to the configuration of your ldap-user-group-provider, that is difficult for me to help with without sample ldap entries for a user and group.  But I'll try to comment on what you shared: <userGroupProvider> <identifier>ldap-user-group-provider</identifier> <class>org.apache.nifi.registry.security.ldap.tenants.LdapUserGroupProvider</class> <property name="Authentication Strategy">LDAPS</property> <property name="Manager DN">CN=ABC1234,OU=Service Users,OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="Manager Password">xxx</property> <property name="TLS - Keystore">/opt/nifi-registry/nifi-registry-current/conf/keystore.jks</property> <property name="TLS - Keystore Password">xxx</property> <property name="TLS - Keystore Type">jks</property> <property name="TLS - Truststore">/opt/nifi-registry/nifi-registry-current/conf/truststore.jks</property> <property name="TLS - Truststore Password">xxx</property> <property name="TLS - Truststore Type">JKS</property> <property name="TLS - Client Auth">NONE</property> <property name="TLS - Protocol">TLS</property> <property name="TLS - Shutdown Gracefully">false</property> <property name="Referral Strategy">FOLLOW</property> <property name="Connect Timeout">10 secs</property> <property name="Read Timeout">10 secs</property> <property name="Url">ldaps://example-v3-prd.corp1.ad1.xyz.net:636</property> <property name="Page Size"/> <property name="Sync Interval">30 mins</property> <property name="Group Membership - Enforce Case Sensitivity">false</property> <property name="User Search Base">OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net</property> <property name="User Object Class">person</property> <property name="User Search Scope">ONE_LEVEL</property> <property name="User Search Filter">(memberof=CN=AB-C-DEF-ADMIN,OU=Groups,DC=corp1,DC=ad1,DC=xyz,DC=net)</property> <property name="User Identity Attribute"/> <property name="User Group Name Attribute"/> <property name="User Group Name Attribute - Referenced Group Attribute"/> <property name="Group Search Base"/> <property name="Group Object Class">group</property> <property name="Group Search Scope">ONE_LEVEL</property> <property name="Group Search Filter"/> <property name="Group Name Attribute"/> <property name="Group Member Attribute"/> <property name="Group Member Attribute - Referenced User Attribute"/> </userGroupProvider> Based on the full DN you configured in the file-user-group-provider, your configuration above is not going to return this user: cn=CEF567,ou=Solid Users,ou=User Accounts,dc=corp1,dc=ad1,dc=xyz,dc=net This is because you have  "User Search Base = OU=User Accounts,DC=corp1,DC=ad1,DC=xyz,DC=net" and "User Search Scope = ONE_LEVEL".  Your user resides at a SUBTREE level under OU=Solid Users", so the User Search scope needs to be SUBTREE instead.   I see that you did not set the User Identity Attribute: this is typically set to the user identity you want to use for the returned user ldap entries.   For AD this is typically "sAMAccountName" and for LDAP, it is commonly "cn". Based on your configuration, you are trying to return all AD/LDAP users that are a memberOf one specific group DN.  I would recommend also setting the "Group Name Attribute to "cn", so that that string is used as the group identity in NiFi rather then the full group DN. I'll stop here since this is a lot just related to the authorizers.xml file configuration. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Shalexey  Your query does not contain a lot of detail around your use case, but i'll try to provide some pointers here. NiFi processor components have one or more defined relationships.  These relationships are where the NiFi FlowFile is routed when a processor completes its execution.  When you assign a processor relationship to more then 1 outbound connection, NiFi will clone the FlowFile how ever many times the relationship usage is duplicated.  So looking at dataflow design shared, I see you have what appears to be  "success" relationship route twice out of the UpdateAttribute processor (this means the original FlowFile is sent to one of these connections and a new cloned FlowFile is sent to the other connection). So you can't simply route both these FlowFiles back to your QueryRecord processor as each would be executed against independently. If I am understanding your use case correctly you ingest a csv file that needs to be updated with an additional new column (primary key).  The value that will go into that new column is fetched from another DB via the ExecuteSQLRecord processor.  Problem is that the ExecuteSQLRecord processor would overwrite your csv content.  So what you need to build is a flow that get the enhancement data (primary key) and adds it to the original csv before the putDataBaseRecord processor. Others might have different solution suggestions, but here is one option that comes to mind: GetFile --> Gets the original CSV file UpdateAttribute --> sets a correlation ID (corrID = ${UUID()}) so that when FlowFile is cloned later both can be correlated to one another with this correlation ID that will be same on both.  ExecuteSQL --> query max key DB QueryRecord --> trim output to just needed max key  ExtractText --> Extract the max key value from the content to a FlowFile Attribute (maxKey). ModifyBytes --> set Remove all Content  to true to clear content from this FlowFile (does not affect FlowFile attributes. MergeContent - min num entries = 2, Correlation Attribute name = corrID, Attribute strategy  = Keep All Unique Attributes.  (This will merge both FlowFiles original and clone with same value in FlowFile attribute "corrID" into one FlowFile containing only the csv content) UpdateRecord --> Used to insert the max key value from the max key FlowFile attribute into the original CSV content. (Record reader can infer schema; however, record writer will need to have a defined schema that includes the new "primary key" column. Then you will be able to add a dynamic property to insert maxkey flowfile attribute into the "primaryKey" csv column. PutDatabaseRecord --> write modified csv to destination DB. Even if this does not match up directly, maybe you will be able to apply the NiFi dataflow design concept above to solution your specific detailed use case. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@sha257  The TLS properties need to be configured if your LDAP endpoint is secured meaning it requires LDAPS or START_TLS authentication strategies.   Even when secured, you will alwasy need the TLS truststore, but may or may not need a TLS keystore (depends on your LDAP setup). For unsecured LDAP url access, the TLS properties are not necessary.  Even unsecured (meaning connection is not encrypted), the manager DN and manager Password are still going to be required to connect to the ldap server. Based on information shared, I cannot say what your ldap setup does or does not require.  You'll need to work with your ldap administrators to understand the requirements for connecting to your ldap. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more