@davehkd  I am not sure I am clear on the ask.  Are you having issues with your 5 node NiFi cluster? As far as certificates go for NiFi, it really does not matter where you obtain them or if you use self-signed (not recommended) as long as the keystore meets the requirements for NiFi. A NiFi node's keystore must meeting the following requirements: 1. Keystore contains only 1 PrivateKey entry.  You can not have multiple PrivateKey Entries in the keystore since NiFi will not know which to use. 2. Keystore PrivateKey entry MUST have Extended Key Usage (EKU) of clientAuth and serverAuth,  NiFi nodes communicate with one another and thus will act as clients and servers in the TLS exchange. 3. Keystore PrivateKey entry must contain a DNS entry for the hostname on which the certificate is being used. A NiFi node's truststore contains 1 too many trustedCertEntries.  It needs to contain the complete trust chain for any client certificates that will be used to authenticate with NiFi via a mutual TLS handshake.  This includes the complete trust chain for each node in yoru cluster.  A trust chain consist of every intermediate CA public cert all the way to the root CA public cert.  The root CA will have the same owner and issuer.  The cacerts file that is included with most java distributions is a truststore containing most public signing authorities intermediate and root CAs. You can obtain a verbose listing of your keystore/truststore using the keytool command found in yoru java install <path to JDK>/bin/keytool -v -list -keystore <keystore or truststore filename> From the output verify following on PrivateKey entry: (DNSName will have your nodes hostname) If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@dyhiamedjouti  It would also be helpful if you shared the full version of Apache NiFi that you installed. The latest versions of NiFi start as secured by default.  The Single User "username" and "password" are only output to the log the very first time NiFi is started.  Subsequent restarts of NiFi service will not log the username and password again. You can stop your NiFi and run the following command to set your own single user identity provider username and password: $ ./bin/nifi.sh set-single-user-credentials <username> <password> Then when NiFi is up and running, you can use your set username and password to access the UI. Not knowing your username and password has nothing to do with the browser not being able to load the NiFi UI for logging in. When you launch NiFi, this starts the NiFi bootstrap process which will then launch a child process which is the main Nifi process.  When this sub process starts, logging will begin in the nifi-app.log.   The NiFi Ui will not be accessible until this process has loaded completely and successfully.  NiFi will log a few lines that contain UI is available at the following URLs.  You'll want to verify you find these log lines and the URLs listed.  These are the URLs you will use in your browser to access your NiFi. If you do not see the URLs output in the logs, that means NiFi failed to successfully start.  Again the nifi-app.log should provide logging details as to why the sub-process failed during the startup process.  Commonly a result of misconfiguration.  If you are not seeing a nifi-app.log produced, then check for the nifi-bootstrap.log for any exceptions. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@davehkd  Unfortunately, I would need to have access to the nifi-app.log file(s) from each node to dig in deeper. Did you copy the flow.xml.gz,  flow.json.gz, users.xml, and authorizations.xml files from NiFi node 1 or 2 to NIFi node 3?  These files all need to match in order for a node to join the cluster.   1. The UI of nifi1 or nifi2  shows "2/2" in the status bar just along top of canvas? 2. The UI of nifi3 shows "1/1" in the status bar just along the top of the canvas? If both above are true, this indicates nifi3 is member of a different cluster. Possible result if issue with your ZK or using a different ZK root node (nifi.zookeeper.root.node).  Check for any leading or trailing whitespace in your configuration. You may also want to inspect your ZK logs for the connections coming from all three nodes. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Matt ... View more

@GSB  If you wanted it always to be two digits, you would need to apply the same if/else NiFi Expression Language (NEL) logic the minute calculations in the working solution provided by @cotopaul  ${value:divide(3600):lt(10):ifElse(${value:divide(3600):prepend(0)},${value:divide(3600)})}:${value:divide(60):mod(60):lt(10):ifElse(${value:divide(3600):mod(60):prepend(0)},${value:divide(3600):mod(60)})} A simpler approach would be to use the toDate and Format NEL functions: ${value:toDate('sssss'):format('HH:mm')} I allow 5 's' assuming that max value would be 86,500 seconds (24 hours in a day) and does not matter if value is smaller.   This format also allows you to quickly and easily adjust format for example, maybe you don't want to truncate the remaining seconds and use ":format('HH:mm:ss')" instead. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@New_User  There is the following resolved jira for the creation of a new putIceberg NiFi processor: https://issues.apache.org/jira/browse/NIFI-10442 Apache has size limits on distributions and it does not appear as though this processor nar was included in the Apache release. You would need to add the nifi-iceberg-processors-nar manually to your NiFi installation for it become available for use. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more

@tkchea  NiFi Remote Process Groups (RPG) transfer FlowFiles and not just the FlowFile content. So depending on the amount of metatdata/attributes on the FlowFile. the amount transferred would be larger. The RPG fetches Site-to-Site (S2S) details via a background thread the runs every 30 seconds regardless of existence of FlowFile.  These S2S details fetched will include details on the target NiFi (Number of nodes in target cluster, load on each node, RAW ports if configured, If HTTP is enabled, etc..).  These details are then used to facilitate the transfer of FlowFiles from client (RPG) and target NiFi (with Remote input or output ports).     The actual transfer of FlowFile will either happen over the HTTPS port (used by a lot of other transactions) or via a RAW socket port depending on configuration. Since a FlowFile consists of two parts (FlowFile Metadata and FlowFile Content), there is going to be disk and CPU I/O involved with writing to the flowfile_repository and content_repository.  So you may want to monitor those on both source and destination.  When it comes to the mutual TLS handshake, NiFi is not doing anything special here.  The client certificate presented is used to identify the client and verify authorization to the send to or pull from a remote port. You can also enable ssl handshake debug logging in the nifi bootstrap.conf file. java.arg.ssldebug=-Djavax.net.debug=ssl,handshake Of course you see all SSL handshakes including those when someone access the NiFi UI in the nifi-bootstrap.log file.   But this would allow you to see if you are seeing systematic slow TLS handshakes or only between these two networks.  You could also setup an RPG that sends to a remote input port on the same NiFi server.  The same TLS handshake will happen there as well.  Is it much faster (rules out an RPG issue.)  If it ends up being the network between NiFi servers, you'll need to investigate there perhaps using something like wireshark may help. Another test might involve using a postHTTP or InvokeHTTP to send to a ListenHTTP or HandleHTTPRequest processor on target server (can be setup to be secure or insecure using same keystore and truststore your NiFi's use). If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt ... View more