About MattWho

MattWho · ‎05-23-2018

@Srini K Based on above comment, it looks like you merged the public key for your private key entries which is not what you shodul have done. - The new truststore you are trying to generate should contain 2 entries Your keystore contains 2 entry Alias name: nifi-cert1 Creation date: May 17, 2018 Entry type: trustedCertEntry Owner: CN=hdfnifidev04.medassurant.local, OU=NIFI Issuer: CN=hdfnifidev04.medassurant.local, OU=NIFI Serial number: 16350a94aa200000000 Valid from: Fri May 11 15:24:22 EDT 2018 until: Mon May 10 15:24:22 EDT 2021 Certificate fingerprints: MD5: A9:BA:53:66:C3:E0:71:EA:94:F0:E2:70:40:F7:B1:58 SHA1: 3C:48:FA:D4:13:C9:CB:6C:D0:74:08:89:32:9B:A3:B9:86:87:0E:49 SHA256: B0:24:27:C3:CA:BB:5F:E7:F7:10:45:62:0E:FC:02:2E:11:08:E8:EA:AA:EB:97:89:B3:63:D9:BF:E7:64:A2:5F Signature algorithm name: SHA256withRSA Version: 3 ..... Alias name: nifi-cert2 Creation date: May 17, 2018 Entry type: trustedCertEntry Owner: CN=rsdevhdf3.medassurant.local, OU=NIFI Issuer: CN=rsdevhdf3.medassurant.local, OU=NIFI Serial number: 162c0b11dfb00000000 Valid from: Fri Apr 13 16:27:35 EDT 2018 until: Mon Apr 12 16:27:35 EDT 2021 Certificate fingerprints: MD5: 0B:E6:9B:74:F3:44:97:93:C6:B6:F7:C2:8D:C4:43:14 SHA1: E0:3F:B4:96:9A:4C:2B:94:AF:85:0D:DA:70:B5:40:B0:AD:B3:9D:4A SHA256: D1:B0:3B:CE:F1:2F:DA:C0:53:49:B9:21:A4:A2:A0:B8:7D:75:78:A1:8C:9E:3A:2B:D0:92:7B:05:66:FE:9E:EF Signature algorithm name: SHA256withRSA Subject Public Key Algorithm: 2048-bit RSA key Version: 3 ...... This new truststore.jks file would need to be copied to both NiFi instance. It would replace the existing truststore.jks being used by those clusters. - Guessing you have used Ambari to install these NiFi instances? **** DO NOT check the box for "NiFi CA Force Regenerate?" <--- Doing so will trigger Ambari to archive the existing keystore.jks and truststore,jks files an run the toolkit to regenerate all new of both. - Note: The NiFi CA (nifi TLS toolkit) was never designed to be used in a production environment. It was designed to facilitate an easy way for users to quickly standup a secured NiFi for testing purposes. - Yes, a NiFi restart will be needed so that NiFi reads the new truststore.jks file you dropped on each NiFi instance. - You do not need to do anything with the keystore.jks files being used by each of your NiFi instances. - Thank you, Matt

MattWho · ‎05-22-2018

@Srini K *** Forum tip: try to avoid responding to an existing answer by starting a new answer. Response may end up being in no specific logical order and become hard to follow. Instead respond to an existing answer by clicking "Add comment" on an existing answer. - Looking at your provide keystore and truststore verbose output, TLS/SSL 2-way authentication is going to fail here. Each of your NiFi server's certificate were signed by different Certificate Authorities (CA). So NiFi-2 cannot trust the client cert being presented from NiFi-1 and NiFi-1 cannot trust the cert being presented from NiFi-2. - You will need to create a new truststore that contain both "trustedCertEntries" (So basically merging both your truststores together). Then configure both NiFi instance to use that new truststore. - Thank you, Matt

MattWho · ‎05-22-2018

@Srini K Can you share the verbose output from your keystore and truststore on each side of this connection? keytool -v -list -keystore <your keystore/truststore jks file> Your keystore should contain only one "PrivateKeyEntry" --- Check that your PrivateKeyEntry has an extended key usage that supports both "ClientAuth" and "ServerAuth" --- Verify that whomever is the "issuer" of your PrivateKeyEntry exists as a trustedCertEntry in the truststore on the other NiFi instance. In above example you can see owner DN is "CN=nifi-sme-14.openstacklocal, OU=NIFI" and it was issued/signed by "CN=nifi-sme-26.openstacklocal, OU=NIFI" - Your truststore would contain 1 to many "trustedCertEntry" --- This file must contain at a minimum a TrustedCertEntry for the issuer of any privateKeyEntry being used to communicate with this system. - You can see here that my truststore contains a trustedCertEntry for the issuer of my cert on the other NiFi. - Thank you, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer.

MattWho · ‎05-22-2018

@Sami Ahmad Moving NiFi's various repositories is an easy process. The nifi.properties file defines for NiFi where to find each one fo the repositories. Specifically look for these lines: - nifi.flowfile.repository.directory=/<my-original-path>/flowfile_repository nifi.content.repository.directory.default=/<my-original-path>/content_repository - Each will be pointing to the current directory path where these repositories reside. 1. Stop your NiFi instance(s). Copy/Move the "content_repository" directory recursively to the new location. 2. Make sure the user that runs your NiFi process has proper ownership and permissions to the new directory location. 3. Update your nifi.properties file to point at new location path: nifi.flowfile.repository.directory=/<my-new-path>/flowfile_repository nifi.content.repository.directory.default=/<my-new-path>/content_repository 4. Start Your NiFi instance(s). - Thank you, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer

MattWho · ‎05-22-2018

@Dilip Namdev The fact that every "archive" sub-directory is empty leads me to believe that archive is in fact working correctly. NiFi stores FlowFile content in claims within the content repository. One claim may contain 1 to many Flowfiles. All it takes is one FlowFile to still be active in one of your dataflows (queued in some NiFi connection) to hold up an entire content claim. A content claim cannot be moved to archive unless all active flowfiles referencing that claim are complete (meaning reached a point of termination in your dataflow). - The following article explains this in more detail: https://community.hortonworks.com/articles/82308/understanding-how-nifis-content-repository-archivi.html - Aside from the above, NiFi opens a lot of file handles. Having insufficient file handles can cause issues with creation of new files. This may affect proper cleanup of both the flowfile and content repositories. I suggest making sure the user that owns your NiFi process has a high number of open file handles available to it. - Thanks, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer

MattWho · ‎05-22-2018

@Dilip Namdev The number of templates you have created will have no impact on NiFi UI performance. When you instantiate a template to the canvas it is merely building a flow like any other flow you would construct. - When it comes to UI performance/responsiveness, a good first place to start is with how many NiFi components you have on your canvas and what state they are currently in. The following article should help you there: https://community.hortonworks.com/articles/184786/hdfnifi-improving-the-performance-of-your-ui.html - Thanks, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer.

MattWho · ‎05-21-2018

@Andrew Riffle If this is just a standalone NiFi installation, change the following property in the nifi.properties file to false: nifi.cluster.is.node=false Then restart your NiFi instance. - Thanks, Matt

MattWho · ‎05-21-2018

@wang ling The "<propertyname="Node Identity 1">CN=rang2, OU=NIFI</property>" property in the authorizers.xml is only used when using NiFi's default file based authorization provider. It does not apply when using Ranger as your authorization provider. - In Ranger, you will need to make sure the user "CN=rang2, OU=NIFI" exists and has been give access to the "/proxy" NiFi Resource Identifier. - Thank you, Matt - If you found this answer addressed your question, please take moment to login and click "accept" below the answer.

MattWho · ‎05-18-2018

@Sharoon Babu NiFi processors like these execute against FlowFiles on inbound connections to the processor. The FlowFile is only removed from the inbound connection when that code execution results in that FlowFile being transitioned to an outbound connection. - There are two types of scenarios here: 1. NiFi is shutdown or dies in the middle of a processors execution. This means the FlowFile was never transferred to an outbound connection. When NiFi is restarted, NiFi will reload FlowFiles in to the last connection they were recorded as belong to. In this case that would be an inbound connection. The consuming processor of that connection will then be scheduled to run/execute again. Processors do not record and intermediate phase fo processing and thus will begin executing against the entire FlowFile again. - 2. Some network failure results in execution not being able to complete. NiFi processors should acknowledge failures in such case which would result in the the FlowFile(s) being moved from the inbound connection to an outbound connection (like a "failure" relationship). It is the responsibility of the dataflow designer to account for such unexpected failures and route those outbound failure relationships accordingly. Often times failure type relationships may be just looped back on the same processor for retry. Wherever this FlowFile is routed (even if in a loop), Execution will again be against the entire Flowfiles content again. - The target systems should handle such scenarios and not except unconfirmed file transfers. - For example: PutFile will write the file using a "dot" rename strategy. The FlowFiles content is originally written as a ".<filename>" and then upon successful completion of writing the data, the filename is renamed from ".<filename>" to just "<filename>". Since dot files are in most cases considered hidden files and ignored by source systems that incomplete transfer would be ignored by destination system. Upon recover and re-attempt (depending on processor configuration) NiFi will repeat this process. - There are some unavoidable scenarios that at times can lead to some data duplication. Considering NiFi's design architecture, NiFi has always favored data duplication over data loss in such rare scenarios. - Thank you, Matt - If you found this answer has addressed your question, please take a moment to log in and click the "accept" link on the answer.

MattWho · ‎05-17-2018

@Takefumi OIDE Additional performance and best practice recommendations: https://community.hortonworks.com/articles/184990/dissecting-the-nifi-connection-heap-usage-and-perf.html https://community.hortonworks.com/articles/184786/hdfnifi-improving-the-performance-of-your-ui.html https://community.hortonworks.com/content/kbentry/109629/how-to-achieve-better-load-balancing-using-nifis-s.html - And just for knowledge relevant to NiFi Content handling: https://community.hortonworks.com/articles/82308/understanding-how-nifis-content-repository-archivi.html

Online	Online
Last Visited	‎02-02-2026 01:13 PM

Member Since	‎07-30-2019 10:41 AM
Last Visited	‎02-02-2026 01:13 PM
Posts	3,432
Kudos received	1628

Cloudera Community

Re: Setting TTL per key when writing to redis

Re: Best Practice for configuring registry flows

Re: Nifi 2.7.2 Start Problem

Re: Error importing NiFi workflow template from ve...

Re: nifi 2.6 registry security scan results

Re: NIFI S2S communication issue between two secur...

Re: NIFI S2S communication issue between two secur...

Re: NIFI S2S communication issue between two secur...

Re: moving nifi repositories

Re: NiFI Content Repository archival is not workin...

Re: NiFI UI Slowness with more than 1000 templates

Re: "Cluster is still in the process of voting on ...

Re: Untrusted proxy CN=rang2, OU=NIFI

Re: What if nifi fails to write data?

Re: HDF/NIFI Best practices for setting up a high ...