Member since
07-30-2019
3387
Posts
1617
Kudos Received
999
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 125 | 11-05-2025 11:01 AM | |
| 375 | 10-20-2025 06:29 AM | |
| 515 | 10-10-2025 08:03 AM | |
| 357 | 10-08-2025 10:52 AM | |
| 394 | 10-08-2025 10:36 AM |
02-06-2025
05:30 AM
@Ghilani The Invalid SNI is caused by the server certificate presented in the TLS exchange not containing a SubjectAlternativeName (SAN) matching the hostname in the request URL. Valid SAN entries are required by the latest version of Java. The solution is to issue new certificates for your NiFi instance(s) that contain all possible SANs used when connecting to the NiFi. So you should have a SAN entry for you NiFi hostname as well as any alternative names used in connection URLs like "nifi" coming form your proxy to NiFi. Please help our community grow. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
02-05-2025
08:51 AM
@Darryl sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target above indicates a trust issue in a TLS handshake. However, there is not enough logging shared to understand what action occurred leading to this exception. The ldap-user-group-provider and ldap-provider configurations you shared has this set: <property name="Authentication Strategy">SIMPLE</property> SIMPLE does not use TLS, so I don't believe this is where your exception is coming from unless you made other configuration changes there. My guess would be a Mutual TLS exchange trust issue between nodes in your NiFi cluster resulting in the exception. No the Toolkit included with NiFi 2 no longer includes the tls-toolkit. This is because there are som many sites and means to create server and client certificates that this really should not be something the done or maintained by the NiFi service. That being said: you could use the tls-toolkit from a NiFi 1.x to create your certificates. You could use an external service to create signed certificates for your NiFi servers. You could create self signed certificates manually yourself. NiFi server certificate minimum requirements are: The Keystore can contain only 1 private key entry. The Private key entry must support both clientAuth and serverAuth ExtendedKeyUsages The private key must contain a SubjectAlternativeName (SAN) entry for the hostname of the server on which the NiFi service is installed. Typically you would create a unique private key for each NiFi node. The truststore can contain 1 too many trusted cert entries (public certificate). But in order for trust to be established between client and server the truststore must contain the trusted cert entry for the signer of the private key of each NiFi host. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
02-04-2025
12:22 PM
@Darryl I understand your use case. I use LDAP on every NiFi install I have, but I am not running on Docker containers. I was highlighting the exception cause and other various configuration issues that will block success with your NiFi configuration setup. As far as making it so your Docker containers can communicate with one another, this discussion in the Docker forum should be able to help you: https://forums.docker.com/t/how-to-create-a-network-of-containers-that-can-communicate-with-each-other-interchangably/134292 Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
02-04-2025
11:38 AM
@Darryl Welcome to the Cloudera Community! From the exception shared the communications exception is Caused by: java.net.UnknownHostException: ldap You have the ldap-user-group-provider configured in your authorizers.xml with: <property name="Url">ldap://ldap:3890</property> But the NiFi host server is unable to resolve "ldap" to an ip address and thus can not communicate with your ldap. On NiFi startup, it will execute the ldap-user-group-provider to pull in your ldap users and groups based on your ldap search configuration and filters. If this provider can't execute successfully, NiFi will not continue to start. Also note that you have the same URL configured in your ldap-provider login-identity-providers.xml file. I see other issues you will hit later once get past the above in your ldap-provider. Also your user search base does not look correct: <property name="User Search Base">CN=users</property> Unfortunately without a sample user ldap entry output, I really can't tell you what the correct value would be for your setup, but the Search Base would typically be as follows: Assume a ldap user has the DN of "cn=John,ou=users,ou=mycompany,ou=nifi". Your Search Base would be "ou=users,ou=mycompany,ou=nifi" In your ldap-user-group-provider, you trying to return all entries with uid=* (The "uid=*" search filter is not necessary here) and then extracting the value from the "cn" field as your user identity for each user returned. However, in your ldap-provider you are expecting users to login with their "sAMAccountName". Do your ldap user entries even have a "sAMAccountName=" field/entry? Assuming your ldap does, yoru ldap-provider is configured to use the user's full DN (<property name="Identity Strategy">USE_DN</property>) as the user's identity. That full DN will not match the user strings being returned by the "cn" field from your ldap-user-group-provider. so you should set the below instead in your ldap-provider: <property name="Identity Strategy">USE_USERNAME</property> This will result in the username entered at the login window being used upon successful authentication as the user identity in NiFi which will be looked up by the authorizer to determine authorizations. And if you are logging in user a user's sAMAccountName, then you should set the below in your ldap-user-group-provider: <property name="User Identity Attribute">sAMAccountName</property> Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
02-03-2025
06:33 AM
@ayu_dev Welcome to the Cloudera Community. Please start a new Cloudera Community question rather then adding a new thread on an existing question from over a year ago. You'll get much better traction there. Your query is unrelated to the original question in this thread around log rotation and retention policies. Fell free to ping me in your new community question and I'll be happy to take a look. Thank you, Matt
... View more
02-03-2025
06:24 AM
@Shampy You are saying that with no flow.json.gz file present, NIFi throws above exception on startup. With No flow.json.gz file, NiFi will create one during startup. I also suggest using the official Apache NiFi 2.x release and stop using the pre-release Maintenance releases that were put out during development of NiFi 2.x. There were 4 maintenance (M1 - M4) releases put out before the first official release of Apache NiFi 2.0.0. The latest official release ofd Apache NiFi 2.x is NiFi 2.2.0 which can be downloaded from here: https://nifi.apache.org/download/ Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-31-2025
09:38 AM
1 Kudo
@mslnrd You are absolutely on the correct track with setting up users and groups in NiFi. Since all synced users and groups are loaded into NiFi's heap memory, it is best practice to limit what is synced to just those users and groups who need to be authorized to access your NiFi. The easiest way to do this is by only syncing the specific groups that contain the users requiring authorized access to your NiFi. For syncing users and groups from AD/LDAP, your authorizers would be configured with the ldap-user-group-provider. Since you can already successfully connect and sync users and groups from your AD/LDAP, i'll just focus on the properties used to control which users and groups are synced: Default settings: <property name="User Search Base"></property>
<property name="User Object Class">person</property>
<property name="User Search Scope">ONE_LEVEL</property>
<property name="User Search Filter"></property>
<property name="User Identity Attribute"></property>
<property name="User Group Name Attribute"></property>
<property name="User Group Name Attribute - Referenced Group Attribute"></property>
<property name="Group Search Base"></property>
<property name="Group Object Class">group</property>
<property name="Group Search Scope">ONE_LEVEL</property>
<property name="Group Search Filter"></property>
<property name="Group Name Attribute"></property>
<property name="Group Member Attribute"></property>
<property name="Group Member Attribute - Referenced User Attribute"></property> First note that it is NOT necessary to configure both the user and group search properties in order to sync both user and group identities. (NOTE: do not unset "class" or "scope" as NiFi will not start) I would recommend the following setup: <property name="User Search Base"></property>
<property name="User Object Class">person</property>
<property name="User Search Scope">SUBTREE</property>
<property name="User Search Filter"></property>
<property name="User Identity Attribute">sAMAccountName</property>
<property name="User Group Name Attribute"></property>
<property name="User Group Name Attribute - Referenced Group Attribute"></property>
<property name="Group Search Base">OU=Groups,DC=my,DC=network,DC=com</property>
<property name="Group Object Class">group</property>
<property name="Group Search Scope">SUBTREE</property>
<property name="Group Search Filter">(|(sAMAccountName=group1)(sAMAccountName=group2)(sAMAccountName=group3))</property>
<property name="Group Name Attribute">sAMAccountName</property>
<property name="Group Member Attribute">member</property>
<property name="Group Member Attribute - Referenced User Attribute"></property> With above configuration the following will happen: A sync of all user will not happen since the user Search Base is not configured. A group sync will happen which will sync only <group1>, <group2>, and <group3> based on he configured Group Search Filter. For each synced group this provider will return all the <member> lines/attributes. For each of those returned members (typically full user DNs), the user will be looked up to obtain the users sAMAccountName identity string (this happens because "sAMAccountName" is configured in the "User Identity Attribute" property. These returned sAMAccountName user identities will be synced in NiFi to the appropriate <group1> or <group2> or <group3> "sAMAccoutName" group identity. Now you can setup Authorizations for either <group1> or <group2> or <group3> for the various NiFi policies. Creating groups in AD/LDAP for the various teams/roles in NIFi allows you to more granular control accesses in NiFi. Once you have authorized your groups any users that are later added to any one of these groups will automatically gain authorized access when the next sync happens in NiFi Default every 30 mins). Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-31-2025
09:12 AM
@doora Welcome to the Cloudera Community! I am not completely clear on the use case described in your query. You have a directory (local directory on the NiFi host or remote directory) whee files are dropped daily. NiFi ingests these files, then parses the content of the file, then deletes the source file, and finally terminates the NiFi FlowFile. Correct? Are you using ListFile and FetchFile to pull these files into NiFi? You want to somehow monitor if a file does not appear in the directory. To do so implies some static naming of these daily files? You could accomplish this through a creative dataflow design if you know the names of the files you are expecting each day. Perhaps using a GenerateFlowFile to create a 0 bytes FlowFile with the filename of the expected file to be fetched within 24 hours and an attribute that captures current time. Configure this processor to run on a cron once a day. This processor would then connect to a FetchFile processor that attempts to fetch that filename from the configured directory. This processor has a not.found relationship which you could connect to a RouteOnAttribute processor which you could configure with two dynamic relationships (one that checks to see if current now minus now recorded by GenerateFlowFile is then 24 hours and another that check if it is greater then 24 hours). The relationship for less then 24 hours would get routed back to FetchFile to check again, The relationship for greater then 24 hours could be routed to perhaps a PutEmail processor to send out an email notifying that filename <xyz> was not found in the past 24 hours t which time you terminate this FlowFile since GenerateFlowFile via it's cron would create a new FlowFile to starting looking for this file in the next 24 hour time window. I would recommend adjusting the run schedule on the RouteOnAttribute to run less often (maybe every 10 minutes) because leaving it at 0 secs will have you FlowFile rapidly looping between FetchFile and RouteOnAttribute until it expires (older then 24 hours) or is found. This would lead to excessive unnecessary resource usage. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-31-2025
08:27 AM
1 Kudo
@mslnrd While kind of related, to prevent confusing other community members, it would be better to start a new community question for this. This question involves proper authorizers.xml configuration and the new question is specific to user-group-provider configuration. Feel free to ping me in that new question so I get notified when it is created. Thank you, Matt
... View more
01-30-2025
06:29 AM
@pbn Welcome to the Cloudera Community. The Apache NiFi community made the decision to remove nifi-toolkit-admin in NIFI-11316 which included: File Manager Flow analyzer Node manager Notify There are no plans to bring these back in NiFi 2.x. NiFi deployments should protect the NiFi repositories from data loss through the use of RAID. Taking backups of the NiFi repositories (mainly flowfile, content, and provenance) makes little sense as the contents of these repositories continuously change. Backup of the config files in the NiFi conf directory and maintaining a copy of the flow.json.gz would allow you to recover a node. Keep in mind that the flow.json.gz is the same on all nodes in a NiFi cluster so it can be restored from any node to a new node. Please help our community grow and thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more