Member since
07-30-2019
3391
Posts
1618
Kudos Received
1000
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 281 | 11-05-2025 11:01 AM | |
| 165 | 11-05-2025 08:01 AM | |
| 502 | 10-20-2025 06:29 AM | |
| 642 | 10-10-2025 08:03 AM | |
| 405 | 10-08-2025 10:52 AM |
11-17-2025
06:48 AM
@Soli Unfortunately, there is not enough shared to say what is taking so long during your startup. Above log snippet shows ~.001 millisecond between each log line. At 34,000 connections, that would be only 34 seconds. Assuming About the same for other components, I don't think the synchronizing of components is what is taking the bulk of the 10-15 minutes of startup time. In the nifi-app.log, what time is reported application start: INFO [main] org.apache.nifi.runtime.Application Started Application in <xxxxx> seconds What does each node report here? Are all nodes taking roughly same time to start or is there any specific node taking longer then the other two? Is performance good once NiFi is up and running? Are all ~21,000 components in the running state? Any WARN or ERROR logging during startup? How many FlowFiles are queued per node? After NiFi is started, maybe collect a verbose NiFi diagnostics output for review. ./bin/nifi.sh diagnostics --verbose diag.txt You may also collect thread dumps every minute during startup to see what thread during is taking so long. Hopefully some of this will help you see what is slowing your startup. There have been some improvements to startup time in Apache NiFi 2.x release. There were a few improvements to startup in Nifi 1.x releases, but those were made prior to 1.22. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
11-13-2025
06:08 AM
@ThierryOfNantes The "install" option for nifi.sh was removed as of NiFi 2.x release because of the nuisances of the many different linux distributions that exist today. I am not implying that is your issue here, but wanted you to be aware that I could be a contributing factor to your issue. Since you can successfully start NiFi manually using the ./bin/nifi.sh start command, NiFi is working fine. So the permission issue is outside of NiFi. What user are you logged in as when you issue the command systemctl start nifi? Is that user able to sudo to your run.as user? Perhaps try enabling debug on your journalctl log to see ifget more detail on where the permission denied is coming from. Hope this helps, Matt
... View more
11-12-2025
07:11 AM
@jfs912 The NiFi toolkit was removed with the NiFi 2.0 releases. See below for more details on why that decision was made. https://lists.apache.org/thread/vn1nzobtz4fh7fs461sgg8jj9zygrk0f I am not well versed on cetic helm charts, so not going to be able to provide specific guidance there. The bottom line is there is nothing special about toolkit generated certficates. I see no reason why you coudl not use the ca from cetic as long as the certificates meet the NiFi requirements for EKUs, SANs, Wildcards.. A NiFi or NiFi-Registry keystore: - Must contain ONLY one PrivateKeyEntry. - PrivateKeyEntry DN must not use wildcards. - PrivateKeyEntry Extended Key Usage (EKU) must support ClientAuth and ServerAuth - PrivateKeyEntry must contain one or more SAN entries. A SAN must match the hostname used by NiFi A NiFi or NiFi-Registry Keystore. (typically same truststore is used by both): - Must contain the complete trust chain for at clientAuth Certificate used to connect with NIFi. This includes any intermediate and root CA in the trust chain. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
11-12-2025
05:31 AM
@casaui Your new issue is completely unrelated to your NiFi to NiFi-Registry MutualTLS authentication based issue in this community question. I kindly ask that your start a new community question so that future users reading this question don't get confuse by multiple unrelated issues being discussed. If you found my assistance on this query helped you, please take a moment to click on "Accept as Solution" on the thread that helped guide you to resolution. Thank you, Matt
... View more
11-06-2025
09:19 AM
@phadkev Have you considered using the ValidateRecord processor instead of the ValidateJson? The ValidateRecord processor can be configured with a JSON reader. The Json readers can be configured to use "Schema Text". The Schema text property support expression language allowing you to extract the Schema you want to use from an inbound FlowFile attribute. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
11-06-2025
06:50 AM
@mirkom If my answer helped you, please take a moment to login and click the "Accept as Solution" button below my response. Thank you, Matt
... View more
11-05-2025
11:01 AM
1 Kudo
@mirkom Yes Parameter Context inheritance does exist now. Been implemented since Apache NiFi 1.15, but had numerous bugs/improvements addressed/implemented since then. From within the "Parameter Contexts" UI, select the parameter context used by Process group and select "edit" from 3 dot menu to far right. Will open UI that looks like this: Select the "Inheritance" tab along the top of this UI. Drag and drop any of the Available Parameter Contexts form the left-hand side to the "selected Parameter Contexts" list on the right-had side. The order in which the Selected Parameter Contexts are ordered in the list is important. The top most takes precedence over those below it. So above shows an example where my "test-param" parameter context is inheriting parameters from the "Root-PG-PC" parameter context. So lets say you have an identical parameter "name" defined in multiple selected parameter contexts. The one defined directly on the PG assigned Parameter context is used. If parameter by that name does not exist there its starts searching inherited parameter contexts from top down. First match found is used. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
11-05-2025
10:41 AM
@Sanga What is the specific Apache NiFi version being used? Any disk space issues on Node-0? Any other exceptions in the NiFi logs? Logs shared cover the reconnection attempt. What was the reason given for the node disconnection? Thank you, Matt
... View more
11-05-2025
08:01 AM
@casaui You have multiple issues with your setup. Issue 1: Problem: Missing ClientAuth certificate in MutualTLS exchange between NiFi's NiFiRegistryFlowRegistryClient and your NiFi-Registry. [NiFi Registry Web Server-50] o.a.n.r.w.s.a.x.X509CertificateExtractor No client certificate found in request. Solution: The NiFiRegistryFlowRegistryClient will use the keystore and truststore defined in the nifi.properties file if a SSL Context service has not been configured in the registry client directly. For a mutualTLS exchange to be successful, the client side must present a clientAuth certificate that is trusted by the truststore being used on the server side (NiFi-Registry) of the TLS connection. Likewise, the serverAuth certificate presented by the server (NiFi-Registry) must be trusted by the truststore the client (NiFiFlowRegistryClient) is configured to use. I have no idea why you have an ALB setup for NiFi-registry since there is just one NiFi-Registry. The NiFiRegistryFlowRegistryClient should be configured with the direct NiFi-Registry URL: https://nifi_ec2server.com:18443/nifi-registry Without a clientAuth certificate presented in the mutualTLS exchange your Registry client is going to be anonymous. Issue 2: Authorization Problem: Every action is authenticated and authorized when it comes to interacting with a secured NiFi or NiFi-Registry. When the Registry Client attempts to establish a connection to the NiFi-registry, it uses the DN from clientAuth certificate from the configured keystore (provided server can trust it) as the client/user identity. From your shared authorizers.xml I can see you are using the managed authorizer --> file-access-policy-provider --> ldap-user-group-provider. So unless your ldap is returning the DN for your clientAuth certificate, you won't be able to setup the necessary authorization needed for the client. Solution: The authorizers.xml could be setup instead to use managed-authorizer --> file-access-policy-provider (modified to use composite-configurable-user-group-provider instead of ldap-user-group-provider) --> composite-configurable-user-group-provider (configured to use configurable provider = file-user-group-provider and provider ldap-user-group-provider). Both the file-user-group-provider and ldap-user-group-provider configured in the composite-configurable-user-group-provider need to also be configured. The file-user-group-provider will allow you manually added client/user identities not provided via your ldap-user-group-provider to which you can set authorization policies needed. The clientAuth certificate DN (may be unique for each NiFi node in a NiFi cluster) would need to be authorized for "Can proxy user requests (read, write, delete)" and "Can Manage Buckets (read)". Issue 3: Authorizers - ldap-user-group-provider use of {0}. Problem: I suspect you are not getting any users returned by this provider because your user search filter is using "sAMAccountName={0}" and group search filter is using "member={0}". "{0}" is going to treated as a literal string. It only has special meaning in the ldap-provider. Solution: Stop using {0} in the authorizers.xml file ldap-user-group-provider. This provider executes every 30 minutes to sync users and groups from ldap. It doe snot take any outside input. The ldap-provider does however. The username entered at login is inserted in place of the {0} in the ldap-provider to be used to verify the username and password provided at login. Issue 4: Authorizers and login-provider - Auth strategy mismatch Problem: You configured auth strategy as Simple; however, configured a ldaps url: <property name="Authentication Strategy">SIMPLE</property>
...
<property name="Url">ldaps://ldapserver.com:636</property> Solution: This provider is not going to work configured like this. Auth strategy should be "LDAPS" when connecting to "ldaps://..." or your Url should be unsecured "ldap://..." if you still want to use "SIMPLE" ------ Your user identity as displayed (case sensitive) in the upper right corner of the NiFi UI, must also exist as a user in NiFi-Registry and be properly authorized on the bucket(s) that user can use for version control. Keep in mind that the NiFiRegistryFlowRegistryClient will proxy read/write request on behalf of your NiFi user. So the authorizations I stated early for the registry client and the authorizations for the NiFi user must both exist in NiFi-Registry. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
11-04-2025
10:16 AM
@pnac03 You will want to restart your NiFi-Registry anytime you modify the configuration of an existing Registry Client or modify the the keystore and/or truststore used by the Registry Client. Only a NiFi restart will trigger the result in those changes being read by the registry client. He is the Apache NIFi Jira that covers this bug: https://issues.apache.org/jira/browse/NIFI-12229 Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more