@vg27  1. So i understand that you have created client certificates for your user.  What authority was used to sign these user certificates?  Was this authority added to the NiFi configured truststore? When you open a browser to NiFi's url, NiFi will respond with a WANT for a clientAuth certificate along with a list of trusted authorities from its truststore.  If your certificate loaded in your browser is  not signed by one of those authorities it will not be presented to NiFi.   If no clientAuth certificate is presented, NiFi will move on to another configured method of user authentication.  The fact that you are seeing  the NiFi login UI, tells me the TLS exchange did not result in a clientAuth certificate being presented by yoru browser. With certificate based mutual Auth there is no login required.  3. "nifi.security.user.login.identity.provider=singleUser" is not a valid configuration. I assume you meant "nifi.security.user.login.identity.provider=single-user-provider.   With "Single-user-provider" configured, the only username and password accepted would be for the single user credentials Nifi auto-generated and output to the logs the first time NiFi was started with that provider configured. If you have no intention of using the single-user-provider, just leave "nifi.security.user.login.identity.provider=" unset. 4. you don't need to worry about sticky sessions if you are only using certificate based authentication, since your client certificate would be passed in every request and their are no tokens involved like in login based providers. If you did decide to use a login-provider like LDAP or Kerberos later, sticky sessions would need to be setup first or you may never be able to access the UI.  Once you enter the username and password, the next request goes is to access UI using that token and if the load balancer were to redirect that to a different node, the UI would not load but instead throw and exception about the unknown user. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Tanya19 @MaxEcueda  The PutIceberg and PutIcebergCDC processors only offer Hadoop or Hive Catalog  Service provider options currently. The only mention of Glue Catalog i could find in an Apache NiFi JIra was the following still open Jira:  https://issues.apache.org/jira/browse/NIFI-11449 It might be a good idea to create an Apache NiFi jira  with as much detail as you can provide around this improvement request for an additional AWS Glue Catalog provider. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@jame1997  1. Everything that is exists on the canvas resides in heap and is persisted to the flow.json.gz file written to the NiFi conf directory (default).  With every change made within the NiFi UI, the current flow.json.gz if moved to an archive directory and new flow.json.gz is generated.  All Nodes in a NiFi cluster use the same flow.json.gz.  You don;t need to stop NiFi to make a copy of the flow.json.gz file.  The NiFi flow.json.gz will contain encrypted values for all sensitive properties entered in any component on the NiFi UI canvas. In order for a NiFi instance to load a flow.json.gz, it must be cnfigured with the same "nifi.senstive.props.key" password used by the NiFi where the flow.json originated.  So always a good idea to backup your NiFi's conf directory (especially the nifi.properties file).  Keep in mind that all nodes in the NiFi Cluster must have same sensitive props key password configured, so unless you lose your entire NiFi cluster, you can get the flow.json.gz an sensitive.props.key form any nodes still accessible. 2. I am not crystal clear on what "backup all the processes" means.  Are you referring to all the dataflows built on the NiFi canvas? Some components (processors, controller services, reporting tasks) may have dependencies on other things like local/cluster state, local files, etc.  So those items need to be taken into consideration with any backup planning.  Each node in a NiFi cluster has a local state directory which hold state for components that use local state (listFile for example).  For other components, cluster state is used (cluster state is written to Zookeeper and Zookeeper quorum maintains protection of that information).  State is always changing as your dataflow is running, so it is not something to easily backup specifically for local state that has not redundancy across NiFi nodes.    Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Kiranq  This error shared: 2024-10-17 08:35:19,764 ERROR [Timer-Driven Process Thread-5] o.a.n.c.s.StandardControllerServiceNode StandardControllerServiceNode[service=CSVRecordLookupService[id=a8b84b00-b0ee-31c8-dbda-7e7e9795ba4b], name=CSVRecordLookupService, active=true] Encountering difficulty enabling. (Validation State is INVALID: ['CSV File' is invalid because CSV File is required, 'Lookup Key Column' is invalid because Lookup Key Column is required]). Will continue trying to enable. Indicates that NiFi is trying to enable a NiFi Controller services loaded from the flow.json.gz during startup, but cannot because it's configuration is invalid.  It is complaining about the configuration of the "CSV File" and "Lookup Key Column" properties. Have you tried starting your NiFi with the following setting in your nifi.properties file set to "false": nifi.flowcontroller.autoResumeState=false This will start NiFi and all components on the canvas will not be started during startup. Also if you NiFi is at the point it is trying to enable components on the canvas, Your NiFi is up and running. As far as the screenshot error, have you verified ownership and permissions on that directory path.  Permissions can be an issue if you started the NiFI service as different users at some point in time resulting in some files created on startup having different ownership.    Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@HenriqueAX  The NiFi nodes themselves require a few authorizations within NiFi registry in order to start version control of a process group in NiFi. The NiFi nodes must be authorized in NiFi-Registry with: 1. "Can proxy user requests" - Read, Write, and Delete 2. "Can manage buckets" - Read Within NiFi you have setup your "Registry Client" that looks like this: For secure (HTTPS) NiFi-Registry URL, when an SSL Context Service is not defined, the default keystore and truststore configured in the NiFi.properties file is used to authenticate with NiFi-Registry through a mutual TLS exchange.  1. Did you setup an SSL Context Service? 2. If so, what keystore and truststore did you configure in the service? 3. Does the truststore used in the nifi-registry.properties file contain the necessary public certificates to include the complete trustchain for your NiFi keystore ClientAuth certificates? From the log snippet shared, it looks like the mutual TLS exchange is not resulting in a trusted clientAuth certificate being passed in the exchange (commonly a trust chain issue). So NiFi-Registry then checks for token in connection which would not exist in a NiFi to NiFi-Registry connection.  So what ends up happening is the connection is established as anonymous (just like you as a user sees when accessing NiFi before clicking login button). 2024-10-17 17:39:54,289 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using X509IdentityProvider 2024-10-17 17:39:54,290 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.x.X509CertificateExtractor No client certificate found in request. 2024-10-17 17:39:54,290 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.IdentityFilter Attempting to extract user credentials using JwtIdentityProvider 2024-10-17 17:39:54,290 DEBUG [NiFi Registry Web Server-15] o.a.n.r.s.a.BearerAuthIdentityProvider HTTP Bearer Auth credentials not present. Not attempting to extract credentials for authentication. 2024-10-17 17:39:54,290 DEBUG [NiFi Registry Web Server-15] o.a.n.r.w.s.a.AnonymousIdentityFilter Set SecurityContextHolder to anonymous SecurityContext So you most likely need to resolve your trust between NiFi and NiFi-Registry.   Typically both NiFi and NiFi-Registry would use the same truststore containing all intermediate and root certificate authorities in the trust chain for the keystore certificates used on both services. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@vg27  Sounds like you may have gotten past your original issue after following the guidance I provided already?  You new questions don't seem directly related to the original query. ------- The invalid host header can be resolved by modifying the nifi.web.proxy.host property in the nifi.properties file:   nifi.web.proxy.host=20.61.182.212,20.61.182.212:9443   Or by including additional entry(s) in your certificates SAN list. --------- Questions: 1. How re you authenticating this user identity?   CN=PAVANBL   2. Have you modified your nif.properties file so that the managed-authorizer is being used?   nifi.security.user.authorizer=managed-authorizer   3. What method(s) or user authentication have you decided to use? Did you just create an clientAuth certificate that you loaded into your browser to authenticate your user or did you setup some other auth method like kerberos or ldap?   nifi.security.user.login.identity.provider= 4. When it comes to using an load balancer, if you are using any type of login provider to authenticate to your NiFi, you need to configure the load-balancer to use session affinity (a.k.a - Sticky sessions).  When you login into a NiFi node the user is issued a client side token and a corresponding server side token is stored on the node.  That client side user token is send in every request made after login.  The issued client token is ONLY valid for use with the NiFi node that issued it.  So session affinity setup in the load balancer is required to make sure that all subsequent request within that same session go to the same NiFi node. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt         ... View more

@Kiranq  @livsey  If you configure "nifi.web.https.host" with your actual Windows hostname or IP rather then using the loopback address (127.0.0.1), does your NiFi start? Any custom add-ons? Have you set NiFi to debug in logback to see if any additional logging shed some light here? Thanks, Matt ... View more

@vg27  A few issues I see with what has been shared: Issue 1: IPV4 addresses like 0.0.0.0 and 127.0.0.1 are special reserved network addresses.  0.0.0.0 is not going to be resolvable to any network host.  0.0.0.0 is typically used by server applications running on multi-homed networks.  Meaning the the server has multiple NIC cards that are connected.  For example: one NIC connected to a companies internal private network and another NIC on same server connected to a public network.  BY configuring the server with 0.0.0.0, the special addresses tells the server to listen and accept connection from all NICs.  From the browser you would still need to use a resolvable IP address for accessing that server via one of the NICs.   NiFi offers a specific configuration property if you want to define multiple network interfaces you want NiFi to listen for connections on. 127.0.0.1 is the loopback address (localhost), which if used in NiFi, means NiFi is only listening on that loopback address.   So you should be configuring your "nifi.web.https.host" property for the hostname of the server on which it is running.  This hostname would need to be resolvable and reachable by your browser.   -------------------------- Issue 2: You are setting up. NiFi cluster, but are still using the single-user providers for authentication and authorization: nifi.security.user.authorizer=single-user-authorizer nifi.security.user.login.identity.provider=single-user-provider These providers were created so that users could launch NiFi standalone instances out-of-the-box for ease of product evaluation.  These providers are not suitable for clustered NiFi setups or multi-users environments. The authorizers.xml you shared is incomplete and only shows a "file-user-group-provider".   The authorizers.xml is easiest to read from bottom of the file upward.  At the very bottom you will find the authorizer (for example: "single-user-authorizer" or "managed-authorizer").   Depending on type of authorizer, the authorizer may utilize one or more additional providers ("file-access-policy-provider", composite-user-group-provider, ldap-user-group-provider", "file-user-group-provider", etc...) already loaded further up in the authorizers.xml.  So while you may have configured the "file-user-group-provider" and the that provider executes on NiFi creating a users.xml file, the configured single-user-authorizer would never use it.   --------------------------- Issue 3: Did you generate your own NiFi keystores and truststore for your 2 NiFi nodes or are you using the NiFi auto-generated keystore and truststore?   I encourage you to manage your own certificates, keystores, and truststores in a production setup.  The certificates generated  automatically are self-signed and only contain very specific Subject Alternative names.  In a cluster setup, NiFi nodes act as both clients and servers since they communicate with one another.  And the owner of those self-signed certifcates is going to be "localhost". This means that all nodes will report as same identity "localhost", which is not good for security. ------------------------------- Issue 4: I see you are using the embedded zookeeper (ZK).  ZK requires quorum in order to be useable.  With only 2 nodes, you only have 2 ZK nodes which is not a quorum.  IF either one of the nodes becomes unreachable or goes down, you will not be able to access the one remaining node either since ZK has lost quorum.   Quorum consists of an odd number of ZK nodes (typically 3 or 5) depending on how much loss tolerance you want to have.   While embedded ZK makes things easy with setup, you'll want to have at least 3 NiFi nodes (this allows you to lose up to one node and still retain access to the NiFi UI of the other 2 nodes.  Of course using an external ZK is even better and the best production option as stopping/starting or loss of NiFi nodes has no impact on quorum. --------------------------------- These are the areas to address first. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more

@Suayb I see no ERROR logging in anything you have shared. The snippet of the nifi-app.log you shared only shows that NiFi is still starting up. NiFi is not fully up and the UI is not accessible until you see the following lines output in the nifi-app.log: 2024-10-17 09:24:32,817 INFO [main] org.apache.nifi.web.server.JettyServer Started Server on https://<hostname>:8443/nifi 2024-10-17 09:24:32,818 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2024-10-17 09:24:32,819 INFO [main] org.apache.nifi.NiFi Started Application Controller in 11.878 seconds (11878131211 ns) Are you seeing above in your nifi-app.log? Are you seeing any ERROR lines in any of you logs? are you seeing any disk space issues? Thanks, Matt ... View more