Member since
07-30-2019
3406
Posts
1622
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 129 | 12-17-2025 05:55 AM | |
| 190 | 12-15-2025 01:29 PM | |
| 131 | 12-15-2025 06:50 AM | |
| 255 | 12-05-2025 08:25 AM | |
| 417 | 12-03-2025 10:21 AM |
06-03-2025
05:55 AM
@lynott @sydney- Are you good now? Your last response contains no questions. If my previous response(s) helped with your initial issue, please a take a moment to click accepted on those responses. Thank you, Matt
... View more
06-03-2025
05:48 AM
1 Kudo
@shiva239 NiFi's DBCPConnectionPool controller services is designed to create a pool of connections that are created the first time it is invoked. These connections can then be used by multiple components on the canvas that are configured to use this same connection pool. This is designed to maximize performance of the dataflows. You can control the behavior of the connection pool using the configuration properties available to this controller service: Max Idle Connections Minimum Evictable Idle Time Minimum Idle Connections Soft Minimum Evictable Idle Time Time Between Eviction Runs Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
06-02-2025
05:18 AM
1 Kudo
@asand3r Your issue is caused by a misconfiguration in the authorizers.xml file here: <userGroupProvider>
<identifier>composite-configurable-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>
<property name="User Group Provider 0">file-user-group-provider</property>
<property name="User Group Provider 1">ldap-user-group-provider-1</property>
<property name="User Group Provider 2">ldap-user-group-provider-2</property>
<property name="User Group Provider 3">ldap-user-group-provider-3</property>
<property name="User Group Provider 4">ldap-user-group-provider-4</property>
</userGroupProvider> The wrong "class" is being used and the wrong property name is being used for the file-user-group-provider. It should look like this: <userGroupProvider>
<identifier>composite-configurable-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class>
<property name="Configurable User Group Provider">file-user-group-provider</property>
<property name="User Group Provider 1">ldap-user-group-provider-1</property>
<property name="User Group Provider 2">ldap-user-group-provider-2</property>
<property name="User Group Provider 3">ldap-user-group-provider-3</property>
<property name="User Group Provider 4">ldap-user-group-provider-4</property>
</userGroupProvider> The "class" needs to be: org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider The above class support one defined "configurable user group provider". A configurable user group provider (file-user-group-provider) is one that allows manual manipulation via the NiFi/NiFi-Registry UI. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-30-2025
06:05 AM
@Ripul Welcome to the Cloudera Community! Sharing a screenshot would be helpful here, but I am assume what you are seeing is something like this when you login with your admin user or other users: This is because of an authorization issue. When NiFi is started for the first time it does not have a flow.josn.gz file yet which contains everything you see on the NiFi canvas. So NiFi will generate that flow.josn.gz which will consist of just a root process group. You'll notice on the cavas the above "Operation" panel. It will show the current selected component on the canvas. With nothing selected on the canvas, it will show details for whichever NiFi Process Group you are currently displaying. Since this is a new install, what the Operation panel is showing is this generated root process group. Anytime you see the name as just the UUID for a component, it indicates the currently authenticated user is not authorized to view that component. A greyed out "gear" (configuration) icon indicates user is not authorized to modify the component. A greyed out "key" (Access Policies) icon indicates currently authenticated user is not authorized to view and maybe modify policies (authorizations) in that component. NiFi provides very granular authorization control all the way down to the individual component level. This may sound like a lot to need to manage; however, there is policy inheritance in place. Example: You add a processor to the canvas. If not explicit policy is defined on the processor itself it will inherit policy from the process group it is inside. If there is no policy defined on the process group, it will inherit policy from parent process group. At the very top level is the above mentioned parent process group. So setting policies on the parent process group will control access on everything added to cavas until ab explicit access policy is set on a sub component. There are also global policies that can be setup and your "admin" user should have been setup on a number of these. From the above global menu found in upper right corner you should see that "Policies" is not greyed out for your admin user. Within global "Policies", all users need to be granted "view the user interface" in order to access the user interface, so it sounds like you have already done this for other users. Your "admin" user should also have "access all policies" (view and modify) which allows that user to view and modify access policies (authorizations) on every component anywhere on the canvas. This policy is what makes the "key" icon not greyed out on the "Operation" panel mentioned earlier. So to give select users (including your admin user) the ability to add components to the root process group, your admin user will need to select the key icon on the root process group and grant those users: Once your admin user and other users are properly authorized to "view the component", the Operate panel will show the process group name instead of just the process group assigned UUID. The gear icon will not be greyed out once your admin user and other users have "modify the component". "Modify the component" on a process group will also allow added users to see the component adding icon a the top of the UI. I am not going to cover all the NiFi Policies, but they can be found in the NiFi Administration guide under Configuring Users & Access Policies Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-29-2025
12:36 PM
@asand3r It is very common to use a load balancer in front of NiFi's UI, but it is necessary to configured Session Affinity (sticky sessions) in the load balancer to make sure all subsequent requests are directed to same node in same session. When you login to a NiFi node using ldap auth, that specific node issues you a user token and stores a corresponding server side token. That server side token is only present on the one node that handled the user authentication. So without session affinity configured redirects within the same session are likely to end up going to other nodes which will reject the client token. The server token is how NiFi controls access expiration since it is not possible to guarantee a client expires and stops using the client token issued. The NiFi node will remove the server side token at expiration or when user clicks logout. You'll also want to make sure that your NiFi node certificates also include a SAN entry for: nifi-service.company.com Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-29-2025
12:21 PM
@lynott I see some issues in your configurations. And have some questions in order to address some other potential issues. Authorizers.xml: File-user-group-provider <identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.file.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Initial User Identity 1">employee</property>
</userGroupProvider> Here you are defining manually a "user identity" of "employee". Is this "employee" a user that is authenticating into your NiFi via ldap authentication. If so, I'd expect this user identity to be returned by the ldap-user-group-provider instead of the file-user-group-provider. You can NOT have two user-group-provider return the same user identity. So if "employee" is going to be returned by ldap-user-group-provider, it needs to be removed from file-user-group-provider. NOTE: The file-user-group-provider will only generate the users.xml if it does not already exist. So modifications to the configuration in the file-user-group-provider will NOT result in any changes to an already existing users.xml. the users.xml file will only contain manually added user and group identities. user and group identities returned by other user-group-provider will not get added to the users.xml. ldap-user-group-provider I assume you ldap connection is good, so I'll just focus on the user sync and group sync areas: <property name="User Search Base"></property>
<property name="User Object Class"></property>
<property name="User Search Scope">SUBTREE</property>
<property name="User Search Filter"></property>
<property name="User Identity Attribute">cn</property>
<property name="User Group Name Attribute"></property>
<property name="User Group Name Attribute - Referenced Group Attribute"></property>
<property name="Group Search Base">OU=COMPANYGroups,DC=corp,DC=company,DC=com</property>
<property name="Group Object Class">group</property>
<property name="Group Search Scope">ONE_LEVEL</property>
<property name="Group Search Filter">((cn=nifi-admins))</property>
<property name="Group Name Attribute">cn</property>
<property name="Group Member Attribute">member</property>
<property name="Group Member Attribute - Referenced User Attribute"></property> As configured you are performing a group sync that will return all the "member" attribute values from the "nifi-admins" group. 1. Are the member attribute values full user DNs? Because you have "cn" configured in the user sync section "user Identity Attribute". each returned "member" is going to be looked up in order to get the cn value. Since NiFi and NiFi-registry user and group identities are case sensitive, it is important that the user identities authorized match exactly with the user identity returned by user authentication in to NiFi-Registry. I see from the nifi-registry.properties file that you are using the ldap-provider for user authentication. Assuming you a re logging in using the user's "cn" as the username, you need to make sure that "USE_USERNAME" is being used and not "USE_DN" in that ldap-provider. Composite-provider: <userGroupProvider>
<identifier>composite-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.CompositeUserGroupProvider</class>
<property name="Configurable User Group Provider">file-user-group-provider</property>
<property name="User Group Provider 1">ldap-user-group-provider</property>
</userGroupProvider> Here you are using the "composite-user-group-provider", but are trying to use a configurable provider with it. Instead you should be using: <userGroupProvider>
<identifier>composite-configurable-user-group-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.CompositeConfigurableUserGroupProvider</class>
<property name="Configurable User Group Provider">file-user-group-provider</property>
<property name="User Group Provider 1">ldap-user-group-provider</property>
</userGroupProvider> You'll notice the "class" is different between these two providers. The "identifier" is not important, but must match the identifier set in your file-access-policy-provider. file-access-policy-prpovider <identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.registry.security.authorization.file.FileAccessPolicyProvider</class>
<property name="User Group Provider">composite-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">employee</property>
<property name="NiFi Group Name"></property>
<!--<property name="NiFi Identity 1"></property>-->
</accessPolicyProvider The file-access-policy provider uses the authorizations.xml file to store all configured authorizations. It will only generate a authorizations.xml file if one does not already exist. It can be configured with an "initial Admin Identity". This configured Initial Admin Identity must be returned by one of yoru user-group-providers. In your case "employee" is being used here. The "NiFi Identity 1", "NiFi Identity 2", "NiFi Identity <xyz>" can be added to define your NiFi node certificate Identities which would allow this provider to seed the proxy policies needed by the NiFi hosts. But keep in mind that the file-user-group-provider would also need to generate these identities for this to work. ------------------- users.xml and authorizations.xml files: Your users.xml only contains one entry for "employee" which corresponds with your authorizers configuration. That user is assigned a UUID. I can then see that UUID properly authorized as and admin user in the authorizations.xml file. What I do not see in the users.xml file is any group defined with uuid: 8c295cae-a773-4d6a-98cd-eef47d0b8189 2. I assume this is the uuid for your ldap-user-group-provider returned "nifi-admins" group? 3. I assume you manually added this group to the authorizations.xml file since you mentioned you have been unable to manually configure user and policies from within the NiFi-Registry UI yet? ---------------------- NiFi-Registry UI: 4. When you login to the NiFi-Registry using user "employee" do you not see "employee" displayed in upper right corner of UI along with the wrench icon? Remember that only case sensitive "employee" and what ever group happens to be "8c295cae-a773-4d6a-98cd-eef47d0b8189" are currently authorized as an admins. 5. I assume you obtained "8c295cae-a773-4d6a-98cd-eef47d0b8189" by enabling debug in the logback.xml for the ldap-user-group-provider class? Or is this the uuid for the NiFi node certificate derived user identity? Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-28-2025
01:37 PM
@lynott When trying to start version control, the NiFi instance will authenticate with NiFi-registry via a MutualTLS exchange. The DN from NiFi's keytore clientAuth privateKey entry is used as the user identity (post NiFi-Registry applying any matching identity.mapping patterns to it). In the request for list of buckets will be the "user identity" displayed in teh upper right corner of NiFi when the request was made. The NiFi node is proxying the request on behalf of that user identity. So If you are not seeing any buckets, either there is a proxy issue for one of your NiFi nodes or the NiFi user identity is not properly authorized in NiFi-Registry for any buckets. Without seeing your NiFi-registry configurations, it would be difficult to say exactly what your issue is. This includes looking at: NiFi-Registry.properties authorizers.xml users.xml authorizations.xml Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-28-2025
07:47 AM
@s198 I am happy to report that I have reviewed CountText with Cloudera's engineering team and Cloudera will officially support the CountText processor effective with CFM 2.1.7 SP2 and CFM 4.10 releases. The documentation has been updated to reflect this change: https://docs.cloudera.com/cfm/2.1.7/release-notes/topics/cfm-supported-processors.html https://docs.cloudera.com/cfm/4.10.0/release-notes/topics/cfm-supported-processors.html Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-28-2025
06:23 AM
@sydney- The exception you shared: "Unable to obtain listing of buckets: javax.net.ssl.SSLHandshakeException:PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target" Indicates a TLS trust issue within the mutualTLS handshake between your NiFi and NiFi-Registry. Perhaps the complete trust chain is missing in the truststores. Looking at the output from OpenSSL may be able to help you see what is missing. openssl s_client -connect <nifi-regsitry-hostname>:<nifi-registry-port> -showcerts
openssl s_client -connect <nifi-hostname>:<nifi-port> -showcerts Need to get past this issue before dealing with and proxied user identity issues. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-28-2025
05:58 AM
@nifier Apache NiFi does not have any AS2 protocol specific processors. This may require building a new processor to handle this protocol. As far as the EDI file format, NiFi is built to be data agnostic so it can handle data of any format. NiFi can do this because it generates a FlowFile that consists of FlowFile content (physical content bytes) and FlowFile attributes/metadata (attributes and metadata about the content on the FlowFile. In this way NiFi can move A FlowFile from NiFi processor to processor with no dependency on content format. Only processors components that need to interact with the content of a FlowFile will need to understand the content's format. This is where the custom processor components may need to created depending on what you are trying to accomplish as part of yoru use case. If you have no need to read or modify your EDI formatted content within your NiFi dataflow(s), then the only custom processor you would need is one that can handle the AS2 file transfer protocol. You could raise a Jiras within the Apache NiFi jira project (https://issues.apache.org/jira/browse/NIFI) with details and request help from that community in developing AS2 protocol processors. If there is enough interest, someone may assist there or you can contribute to the Apache NiFi project yourself. You'll need to create an account before you can create a a new jira in that project. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more