Member since
05-13-2025
7
Posts
0
Kudos Received
0
Solutions
07-29-2025
08:21 AM
Hello, Unfortunately, there is no version of Ambari, supported by Cloudera, that has log4j1 removed or fixed as it was for log4j2. That script was made as a temp workaround while we upgraded log4j2 in our products. The log4j1 package is used by almost all components. It can not be removed. Cloudera also does not support removing any classes from the JAR as they may be used by the services as well and it has never been vetted or verified. The proper path here is an upgrade to a current version of CDP where Log4j has been completely replaced with reload4j. If you have any questions please contact us via one of the methods listed here: https://www.cloudera.com/contact-sales.html Regards, Cloudera Support
... View more
06-23-2025
09:04 AM
@melek6199 What you have is an authorization issue. When you access you multi-node NiFi cluster, you are authorized only into the node in which you authenticated. When you make a request like List Queue or Empty Queue, you are making a request from one node to all the other nodes to list or empty the connection queue. This means that the nodes themselves need to be authorized to request other nodes to share back their queue list or empty their target node queues. All 4 of your NiFi nodes should already have been authorized for "proxy user requests", but in order to list or empty queues, your node will need these additional authorizations: "view the data" - authorizes a node(s) to list the data from other nodes (user must also be authorized) "modify the data" - authorizes a node(s) to empty a connection queue on other nodes. You can see from the nfi-user.log output you shared the identity and policy missing to perform this action on the specific connection UUID: Node x.x.x.x:8443 is unable to fulfill this request due to: Unable to modify the data for Processor with ID d3a802c6-0196-1000-ffff-ffff90fdc7b8 You would have seen this same exception for all but one node when you made the request to empty the queue. Authorizations are inherited form parent Process groups unless explicitly set on the individual component directly. So you don't need to authorize your nodes for "view the data" and "modify the data" on the connection "d3a802c6-0196-1000-ffff-ffff90fdc7b8" directly, but rather set these authorization instead on the parent process group. Keep in mind that child process groups also inherit from parent process groups unless policy is explicitly set on that child process group.. Typically you would set these authorization policies on the root process group (top level). You'll also notice when you are viewing policies on a component it will tell you if it is inheriting policies and if you choose to set explicit policies on that component it asks you if you want to copy the inherited policy before modifying. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-21-2025
06:32 AM
@melek6199 Let me try to address each of your statements as there appears to be some misunderstanding of how authentication and authorization works between NiFi and NiFi-Registry. I have a 3-node NiFi cluster, and I want to manage it using NiFi Registry. I configured both NiFi and NiFi Registry with a single certificate using the TLS Toolkit. I also set up LDAP integration. I can successfully connect to both NiFi and NiFi Registry individually using my LDAP users. NiFi-Registry does not manage your NiFi cluster. These are two different services. NiFi-Registry is used by NiFi to version control process groups created and managed in NiFi. It is not a security best practice to use one certificate for all your servers. You should have one certificate per server. If you have two services (A NiFi node and NiFi-Registry on the same server, they would both use the same certificate). In production I would recommend using certificates signed by actual legitimate signing authorities versus TLS toolkit generated certificates and truststore) Your keystores must meet the following requirements: Contain only one PrivateKey Entry That PrivateKey entry supports both ClientAuth and ServerAuth ExtendedKeyUsage (EKU). (Note: NiFi-Registry does not require ClientAuth, but no harm in having it) Contains at least one SAN entry that matches the server's hostname on which the certificate is being used. Your NiFi/NiFi-Registry Truststore must meet following requirements: Contain a TrustedCertEntry for every signer/issuer of the certificates passed in a mutualTLS handshake (The compete trustchain for every certificate that will be used to communicate between NiFi node and with NiFi-Registry). You can use the NiFi TLS toolkit to generate 4 keystores and 1 truststore you can use with your NiFi and NiFi-Registry services, but make sure you are running with the "--subjectAlternativeNames" option. Those SAN should include the hostnames of the servers on which the services will run. (Now technically you could create one certificate with SANs for all the hosts and then use that one cert on all hosts, but as I said, not a security best practice). However, the LDAP user that I added and authorized in the Registry does not appear in NiFi. With the certificate user, I can view the bucket in NiFi Registry from NiFi and perform flow version control. But I cannot do this with my LDAP user. The user that authenticates into NiFi-Registry does not need to exist in NiFi; however, any authenticated user identity authenticated into NiFi must exist and have proper authorization in NiFi-Registry in order to conduct version control operation within NiFi. When you ldap-user authenticates into NiFi you will see that user's "user identity" displayed in the upper right corner (Keep in mind that your user is only authenticated into the NiFi node you access the cluster from and not all the NiFi nodes). When that user attempts to start version control on a process group, NiFi connects and authenticates with NiFi-Registry via a MutualTLS exchange/handshake. In that connection it will proxy the request on behalf of that "user identity" (case sensitive). This means that not only do the NiFi node clientAuth certificates need to be authorized in NiFi-Registry to read on "Can Manage Buckets " and read,write,delete on "Can Proxy Requests", the NiFi "user identity" need to be authorized on any bucket you want that "user identity" to be able to use for version control. (let me know if you need help with how a mutualTLS handshake works) Since your NiFi authenticated ldap "user-identity" has not been added and authorized in NiFi-Registry on any buckets, nothing will appear in the list of available buckets for that "user-identity" in NiFi. NOTE: Even if I generate separate certificates for NiFi and NiFi Registry and trust each certificate independently, the certificate user does not have permission to view the bucket. This is because the certificate user from the Registry is also not created in NiFi. For this reason, I generated both from the same certificate. From what i shared in response to two section above, you can see that the certificates used by the NiFi host are only used to proxy requests to NiFi-Registry on behalf of the "user identity" authenticated with NiFi. There are few things that don't make sense to me in your shared NiFi-Registry configuration: NiFi-Registry identity-providers.xml: I see you set " <property name="Authentication Strategy">SIMPLE</property>", yet your ldap URL is "ldaps". This should then also be set to "LDAPS". NiFi-Registry authorizers.xml: You are using the file-user-group-provider which allows you to manually define an initial set of "user identities" on first startup (node edits after to this config will happen if the users.xml already exists during startup). This provider also allows for the adding of additional "user identities" later via the NiFi-Registry UI directly. NOTE: There is also an available ldap-user-group-provider that can be used to sync select users "user identity" and groups "group-identity" from ldap into your NiFi-Registry list of identities. This is helpful if you don;t want to manage your ldap user and group identities manually within NiFi and NIFi-Registry. You are the file-access-policy provider which only created the authorizations.xml file if it does not already exist on startup. In it I can see "Initial Admin Identity">CN=nifi_amadeus_admin is set; however, in your ldap-provider you have configured "Identity Strategy">USE_USERNAME. I can only assume you did similar in your NiFi setup? It is unlikely that when you are logging into your NiFi you are typing the username as "CN=nifi_amadeua_admin" since this would not be the expected value in the "sAMAccountName" ldap field/attribute. That means your initial admin "user identity" does not match the identity of your authenticated user (unless you have this set because you are using the a certificate to auth in to the services with the above). In the end, there are the following key things that need to know: user-identities must match and are case sensitive. ("Bob" and "bob" would be treated as two unique user identities. So the user identity as displayed in upper right corner in NiFi UI must be authorized on specific bucket(s) in NiFi-Registry in order to successfully use version control in NIFi. This you do not have setup correctly yet. NiFi nodes also need to be properly authorized in NiFi-Registry for mange buckets and all proxy permissions. The node's user identity is comes from the NiFi node's clientAuth certificate full DN. That full DN can be modified through the use of the identity.mapping properties in the NiFi-Registry.properties file. Note: In your shared NiFi-Registry.properties file the identity.mapping properties are commented out are not in use, so full DN of NiFi node would be used as node's user identity and need to be authorized which corresponds to with full DN used in the file-access-policy provider you have configured. I know above is a lot of information, but wanted you to fully understand how the authentication and authorization between NiFi and NiFi-Registry works. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
05-13-2025
07:01 AM
@melek6199 When you setup an Apache NiFi cluster versus a standalone NiFi instance, the cluster coordinator and zookeeper become part of the setup. Since a NiFi cluster is a zero master cluster, the UI can be access from any cluster connected node. So your user authenticates to the specific node you are accessing and then that node proxies the user request (initially that would "access the flow") on behalf of that user to the cluster coordinator that replicates request to all connected nodes. The exception means that node with node identity derived from certificate DN "CN=vtmrt3anifit04.x.com, OU=NIFI" was not properly authorized to "proxy user requests". All your NiFi node identities must be authorized to "proxy user requests". While it appears that your NiFi authorizers.xml is setup correctly with your 4 node's identities (case sensitivity also correct), I suspect it was only setup correctly after NiFi having already being started before it was configured correctly. The "file-access-policy-provider" will only generate the authorizations.xml during NiFi startup if it does NOT already exist. It also will not modify an already existing authorizations.xml file. The "file-user-group-provider" will only generate the users.xml during NiFi startup if it does not already exist. It also will NOT modify an already existing users.xml file. So I would inspect the users.xml to make sure it contains all 4 node identities (case sensitive correctly) and then verify the authorizations.xml has those node's properly authorized. So I would start here to make sure above is correct on all 4 nodes. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more