Member since
07-30-2019
3406
Posts
1622
Kudos Received
1008
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 98 | 12-17-2025 05:55 AM | |
| 159 | 12-15-2025 01:29 PM | |
| 104 | 12-15-2025 06:50 AM | |
| 238 | 12-05-2025 08:25 AM | |
| 398 | 12-03-2025 10:21 AM |
05-23-2018
08:26 PM
@Srini K Managing TLS/SSL issued certificates is not a job for NiFi. The NIFi CA is not backed by any type of long time management capability. Is becomes complicated to deploy and manage across multiple NiFi deployments. You are experiencing some of that management difficulty here now. - 1. Every Ambari based NiFi installation sets up its own CA. So it becomes difficult to setup communication between systems where each has a different Unique CA signing their certificates. You are trying to get two NiFi's to talk to one another, but this difficulty would extend to any other external service that NiFi needs to communicate to or receive connections from over TLS/SSL. 2. There is no build in management process to handle certificate renews or notifications of expiration. Your NiFi system may be working fine one day in production and then stop working all together the next because your certs expired. - In a production managed environment, a corporately or external managed CA should be used to issue, sign, and manage all your certificate needs. - Yes, NiFi requires TLS/SSL certificates in order to secure NiFi, but SSL/TLS is not a product of NiFi. - As far as merging the content of your two truststore in to a new truststore... A truststore cannot contain multiple keys with the same alias. Each entry must have a unique alias. The trustedCertEntries in each of your existing truststores have the same alias, so you are going to need to change the alias of one of them. Following commands will extract the trustedCertEntry from each source truststore.jks and put tehem in a new-truststore.jks with new unique alias fro each of those entries: keytool -importkeystore -srckeystore truststore.jks -destkeystore new-truststore.jks -srcalias nifi-cert -destalias nifi-cert1 -srcstorepass ****-deststorepass **** keytool -importkeystore -srckeystore truststore.jks -destkeystore new-truststore.jks -srcalias nifi-cert -destalias nifi-cert2 -srcstorepass ****-deststorepass ****
... View more
05-23-2018
08:04 PM
@Zack Riesland Hope I know NiFi, been working with it almost since the beginning (8+ years now) 🙂 - The only port that needs to be open for users to access the NiFi UI of an unsecured NiFi is which ever port is defined in the nifi.web.http.port= property in the nifi.properties file. - That being said, is your server hosting NIFi multi-homed? The nifi.web.http.host= property is typically configured with the FQDN for your host. When left blank NiFi will bind to every interface on your host. When set it will bind to only the interface which the FQDN is assign to. So maybe you are having issues because NiFi has bound to an interface you cannot reach from other computers on your network? - During startup you will see a line that says: org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: immediately following that line will be a list of URLs that can be used to access this running NiFi instance. - Some components in NiFi that create their own listening port (example: ListenHTTP) would have a unique port that would need to be opened before they could be accessed by external systems as well - Thanks, Matt
... View more
05-23-2018
04:55 PM
@Zack Riesland ***Forum Tip: Try to avoid responding to an existing "Answer" by starting a new "Answer". There is no guaranteed order to answer, so conversation may get difficult to follow. Instead use the "Add comment" below an answer to respond. - That works as long as you have discipline amongst your various devs. With NiFi being unsecured there will be no controls preventing one user from modifying another users flows or deleting other users components. There also will not be an audit trail, as every change is logged as being done by anonymous. - Otherwise, a process group does provide user with what appears to be their own work space. - Keep in mind that it is still a single NiFi JVM, so every dataflow (no matter whom built it) is operating unders the same shared resource constraints (CPU, HEAP, etc..). - Thanks, Matt
... View more
05-23-2018
04:33 PM
@Zack Riesland Multiple users can access and make changes on the canvas at the same time. The only time that acton is blocked is if both users are trying to edit the exact same component at the same time. First user in such a scenario to hit "accept" on their changes wins. The other users change will be lost and they will be force to refresh and try again. - When NiFi is secured, you have the ability to setup very granular access controls. You can restrict different authenticated users for example to only have access to specific process groups. This would prevent users from editing components belonging to other users. - Thanks, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer.
... View more
05-23-2018
04:28 PM
@Srini K Based on above comment, it looks like you merged the public key for your private key entries which is not what you shodul have done. - The new truststore you are trying to generate should contain 2 entries Your keystore contains 2 entry
Alias name: nifi-cert1
Creation date: May 17, 2018
Entry type: trustedCertEntry
Owner: CN=hdfnifidev04.medassurant.local, OU=NIFI
Issuer: CN=hdfnifidev04.medassurant.local, OU=NIFI
Serial number: 16350a94aa200000000
Valid from: Fri May 11 15:24:22 EDT 2018 until: Mon May 10 15:24:22 EDT 2021
Certificate fingerprints:
MD5: A9:BA:53:66:C3:E0:71:EA:94:F0:E2:70:40:F7:B1:58
SHA1: 3C:48:FA:D4:13:C9:CB:6C:D0:74:08:89:32:9B:A3:B9:86:87:0E:49
SHA256: B0:24:27:C3:CA:BB:5F:E7:F7:10:45:62:0E:FC:02:2E:11:08:E8:EA:AA:EB:97:89:B3:63:D9:BF:E7:64:A2:5F
Signature algorithm name: SHA256withRSA
Version: 3 .....
Alias name: nifi-cert2
Creation date: May 17, 2018
Entry type: trustedCertEntry
Owner: CN=rsdevhdf3.medassurant.local, OU=NIFI
Issuer: CN=rsdevhdf3.medassurant.local, OU=NIFI
Serial number: 162c0b11dfb00000000
Valid from: Fri Apr 13 16:27:35 EDT 2018 until: Mon Apr 12 16:27:35 EDT 2021
Certificate fingerprints:
MD5: 0B:E6:9B:74:F3:44:97:93:C6:B6:F7:C2:8D:C4:43:14
SHA1: E0:3F:B4:96:9A:4C:2B:94:AF:85:0D:DA:70:B5:40:B0:AD:B3:9D:4A
SHA256: D1:B0:3B:CE:F1:2F:DA:C0:53:49:B9:21:A4:A2:A0:B8:7D:75:78:A1:8C:9E:3A:2B:D0:92:7B:05:66:FE:9E:EF
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
...... This new truststore.jks file would need to be copied to both NiFi instance. It would replace the existing truststore.jks being used by those clusters. - Guessing you have used Ambari to install these NiFi instances? **** DO NOT check the box for "NiFi CA Force Regenerate?" <--- Doing so will trigger Ambari to archive the existing keystore.jks and truststore,jks files an run the toolkit to regenerate all new of both. - Note: The NiFi CA (nifi TLS toolkit) was never designed to be used in a production environment. It was designed to facilitate an easy way for users to quickly standup a secured NiFi for testing purposes. - Yes, a NiFi restart will be needed so that NiFi reads the new truststore.jks file you dropped on each NiFi instance. - You do not need to do anything with the keystore.jks files being used by each of your NiFi instances. - Thank you, Matt
... View more
05-22-2018
09:32 PM
@Srini K *** Forum tip: try to avoid responding to an existing answer by starting a new answer. Response may end up being in no specific logical order and become hard to follow. Instead respond to an existing answer by clicking "Add comment" on an existing answer. - Looking at your provide keystore and truststore verbose output, TLS/SSL 2-way authentication is going to fail here. Each of your NiFi server's certificate were signed by different Certificate Authorities (CA). So NiFi-2 cannot trust the client cert being presented from NiFi-1 and NiFi-1 cannot trust the cert being presented from NiFi-2. - You will need to create a new truststore that contain both "trustedCertEntries" (So basically merging both your truststores together). Then configure both NiFi instance to use that new truststore. - Thank you, Matt
... View more
05-22-2018
06:27 PM
1 Kudo
@Srini K Can you share the verbose output from your keystore and truststore on each side of this connection? keytool -v -list -keystore <your keystore/truststore jks file> Your keystore should contain only one "PrivateKeyEntry" --- Check that your PrivateKeyEntry has an extended key usage that supports both "ClientAuth" and "ServerAuth" --- Verify that whomever is the "issuer" of your PrivateKeyEntry exists as a trustedCertEntry in the truststore on the other NiFi instance. In above example you can see owner DN is "CN=nifi-sme-14.openstacklocal, OU=NIFI" and it was issued/signed by "CN=nifi-sme-26.openstacklocal, OU=NIFI" - Your truststore would contain 1 to many "trustedCertEntry" --- This file must contain at a minimum a TrustedCertEntry for the issuer of any privateKeyEntry being used to communicate with this system. - You can see here that my truststore contains a trustedCertEntry for the issuer of my cert on the other NiFi. - Thank you, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer.
... View more
05-22-2018
04:22 PM
2 Kudos
@Sami Ahmad Moving NiFi's various repositories is an easy process. The nifi.properties file defines for NiFi where to find each one fo the repositories. Specifically look for these lines: - nifi.flowfile.repository.directory=/<my-original-path>/flowfile_repository
nifi.content.repository.directory.default=/<my-original-path>/content_repository - Each will be pointing to the current directory path where these repositories reside. 1. Stop your NiFi instance(s). Copy/Move the "content_repository" directory recursively to the new location. 2. Make sure the user that runs your NiFi process has proper ownership and permissions to the new directory location. 3. Update your nifi.properties file to point at new location path: nifi.flowfile.repository.directory=/<my-new-path>/flowfile_repository
nifi.content.repository.directory.default=/<my-new-path>/content_repository 4. Start Your NiFi instance(s). - Thank you, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer
... View more
05-22-2018
04:02 PM
1 Kudo
@Dilip Namdev
The fact that every "archive" sub-directory is empty leads me to believe that archive is in fact working correctly. NiFi stores FlowFile content in claims within the content repository. One claim may contain 1 to many Flowfiles. All it takes is one FlowFile to still be active in one of your dataflows (queued in some NiFi connection) to hold up an entire content claim. A content claim cannot be moved to archive unless all active flowfiles referencing that claim are complete (meaning reached a point of termination in your dataflow). - The following article explains this in more detail: https://community.hortonworks.com/articles/82308/understanding-how-nifis-content-repository-archivi.html - Aside from the above, NiFi opens a lot of file handles. Having insufficient file handles can cause issues with creation of new files. This may affect proper cleanup of both the flowfile and content repositories. I suggest making sure the user that owns your NiFi process has a high number of open file handles available to it. - Thanks, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer
... View more
05-22-2018
03:51 PM
@Dilip Namdev The number of templates you have created will have no impact on NiFi UI performance. When you instantiate a template to the canvas it is merely building a flow like any other flow you would construct. - When it comes to UI performance/responsiveness, a good first place to start is with how many NiFi components you have on your canvas and what state they are currently in. The following article should help you there: https://community.hortonworks.com/articles/184786/hdfnifi-improving-the-performance-of-your-ui.html - Thanks, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer.
... View more