Member since
07-30-2019
3436
Posts
1632
Kudos Received
1012
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 138 | 01-27-2026 12:46 PM | |
| 559 | 01-13-2026 11:14 AM | |
| 1206 | 01-09-2026 06:58 AM | |
| 1004 | 12-17-2025 05:55 AM | |
| 494 | 12-17-2025 05:34 AM |
06-14-2021
09:25 AM
1 Kudo
I think I figured out with this issue on my side and I share my results with you. I deal with very small and very huge contents, the default values for properties hereafter are not appropriate (Claim size retained ...), so I decrease drastically the values as follows: nifi.content.claim.max.appendable.size=10 MB ==> 5MB nifi.content.claim.max.flow.files=100 ==> 50 nifi.content.repository.archive.max.retention.period=12 hours ==> 2 hours Until now, the platform doesn't freeze anymore. Jean-Louis
... View more
06-14-2021
06:20 AM
@Rupesh_Raghani NiFi was not designed to provide a completely blank canvas to each user. There are important design reason for this. NiFi runs within a single JVM. All dataflows created on the canvas run as the NiFi service user and not as the user who is logged in. This means that all user's dataflows share and compete for the same system resources. Another user's poorly designed dataflow(s) can have an impact on the operation of another user's dataflow(s). So it is important for one users to be able to identify where backlogs may be forming even if that is occurring in another user's dataflow(s). With a secured NiFi, authorization policy control what a successfully authenticated user can see and do on the NiFi canvas. While components added to the canvas will always be visible to all users, what is displayed on the component is limited only stats for unauthorized users (no component names, component types, component configurations, etc). So an unauthorized user would be unable to see how that unauthorized component is being used and for what. The unauthorized user would also not have access to modify the component, access FlowFiles that traversed those components (unless that data passed through an authorized component somewhere else in the dataflow(s)), etc. Besides resource usage, another reason users need to see these place holders for all components is so that users do not build dataflows atop one another. It is common for multiple teams to be authorized to work within the same NiFi. It is also common to have some users who are members of more than one team. For those users, it would be very difficult to use the UI if each teams flows were built on top of one another. Most common setup involves an admin user creating a single Process Group (PG) on the root canvas level (top level - what you see when you first log in to a new NiFi). Then each team is authorized only to their assigned PG. So team1 user logs in and there PG is fully rendered and non authorized PGs are present by non configurable and no displayed details. team1 is unable to add components to canvas at this level and must enter their authorized PG before they can start building dataflows. When you enter sub-PG, you have a blank canvas to work with. Hope this helps with your query. Matt
... View more
06-13-2021
10:58 PM
@AnkushKoul , did @MattWho's response resolve your issue? If so, can you please mark it as the solution? It will make it easier for others to find the answer in the future.
... View more
06-13-2021
10:55 PM
@midee, did you resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
06-08-2021
06:41 AM
@ang_coder The RouteOnAttribute processor establishes a NEW relationship for each dynamic property you add. If you intent is that a single FlowFile must satisfy all conditions to route on, then you should have just one NiFi Expression Language (NEL) statement that covers all conditions resulting a true or false boolean. If you share your two statements, I'd be happy to help you construct a single NEL statement. butt would be structured something like something like: All 4 lines would need to result in true before FlowFile would be related to this dynamic property's relationship. If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-08-2021
06:07 AM
@techNerd The PutSFTP processor contains the following configuration property: Do you have that set to false on the particular putSFTP processor throwing the exception? Thanks, Matt
... View more
06-07-2021
09:30 AM
@Chakr So first thing to check is proper configuration of your Authorizers.xml file. The order in which the sections are added is very important. The file is loaded from top down, so you can NOT have a section refer to another section that is not yet loaded. And you users need to be loaded/created before the authorizations can be loaded/created. So your authorizers.xml file structure should look like this: <authorizers>
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">CN=chakri, OU=NIFI</property>
<property name="Initial User Identity 2">CN=nifi-1,OU=NIFI</property>
<property name="Initial User Identity 3">CN=nifi-2,OU=NIFI</property>
<property name="Initial User Identity 4">CN=nifi-3,OU=NIFI</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">CN=chakri, OU=NIFI</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1">CN=nifi-1,OU=NIFI</property>
<property name="Node Identity 2">CN=nifi-2,OU=NIFI</property>
<property name="Node Identity 3">CN=nifi-3,OU=NIFI</property>
<property name="Node Group"></property>
</accessPolicyProvider>
<authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
</authorizers> Note: Changed your authorizer to the newer "StandardManagedAuthorizer". On NiFi startup the "file-user-group-provider" will create the users.xml file only if it does NOT already exist and seed it with all four configured "Initial User Identity" strings. These strings (case sensitive) will each be assigned a UUID. On NIFi Startup the "file-access-policy-provider" will create the authorizations.xml file only if it does NOT already exist and seed it with relevant policies needed initially for your admin user and your nodes based off the UUIDs create for each user identity generated by the first provider. The the Authorizer is loaded which reads in the users and authorizations from the users.xml and authorizations.xml files that are now created. You should do the following: 1. Edit your authorizers.xml to match above structure above. 2. In your nifi.properties file, make sure following property is set correctly as: nifi.security.user.authorizer=managed-authorizer 3. Remove/rename current users.xml and authorizations.xml files. 4. Start NiFi You should not be able to successfully access your NiFi's UI. If you fin this addresses your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-07-2021
08:58 AM
@Pe Communications between a secured NiFi-Registry and a Secured NiFi cluster node or standalone NiFi instance will use a mutual TLS connection. At a very high level... - That means the client (NiFi node) will initiate a connection to the server (NiFi-Registry). T - The NiFi-Registry will responded via that handshake and present its serverAuth certificate along with a list of trusted entities (this would be the list of DNs from the trustedCertEntries found in the NiFi-Registry's truststore file). - The Client NiFi node will verify that it trusts the server certificate presented. This means the connecting NiFi node's truststore must contain the complete trust chain for the NiFi-registry's privateKeyEntry certificate. - The client will then respond to that server Hello with a client hello that includes the clientAuth certificate from that NiFi nodes keystore file (Only way it will send a client certificate is if that client certificate was signed by by one of the DNs presented from the NiFi-Registry. No sense sending a client certificate back to NiFi-Registry, if NiFi-Registry is unable to trust it). - The NiFi-Registry will then use that DN from the NiFi client certificate to identify the connecting NiFi instance. - NiFi-Registry will then check that that connecting NiFi node is authorized properly. Now many setting up additional NiFi clusters or instances, the same applies to each of those NiFi nodes when communicating to this one NiFi-Registry. - So truststore on every NiFi node must contain the complete trust chain for the NiFi-Registry's certificate. - The NiiFi-Registry's trustsstore file must contain the complete trust chain for the NiFi instances keystore files certificate. (Each NiFi node should have its own unique keystore file that contains only 1 PrivateKeyEntry) - It is very common to create 1 truststore file with every needed trustedCertEntry (public certificates) and then distributed that one truststore to all instance of NiFi and NiFi-Registry. NOTE: The NiFi keystore files must meet following criteria. 1. Must contain ONLY 1 PrivateKeyEntry. Having more than 1 PrivateKeyEntry will not work as NiFi will not know which to use. 2. The DN used in the PrivateKeyEntry must not contain wildcards. Since NiFi certificate is used for ClientAuth, the PrivateKeyEntry DN is what is presented to identify the node. Many Authorizers will not support client names with wildcards, plus it is not advisable security wise. 3. The PrivateKeyEntry must have an Extended Key Usage (EKU) that supports both clientAuth and serverAuth 4. The PrivateKeyEntry must have at least 1 SAN entry that matches the hostname for the server on which the keystore is being used. If you found this assisted with your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-07-2021
08:34 AM
@Acbx It looks like your CSV uses commas as the field delimiter. So the solution i provided parses the entire file line by line and changes all "." to ",". So, I am guessing that you have other places within your CSV that also had ".", thus creating the additional 5 field columns. Are trying to create a new column for cents? Is that why you are changing 109.29 tp 109,29? If you are not looking for a new column, how will downstream system parse this edited CSV now that you added a new comma in there? You could write a complex Java regular expression in the Search Value to match only specifically on column number X (Money Column) and then use Replacement Strategy "Regex Replace" to edit it. Let's assume the "Money" Column was column number 5. And then wrap money once converted from 109.29 to 109,29 in quotes so it is not treated as two columns later on.... Search Value: ^(.*?),(.*?),(.*?),(.*?),(.*?),(.*?)$ Replacement Value: $1,$2,$3,$4,"${'$5':replace(".",",")}",$6 So above would manipulate column 5 only and change 109.29 in to "109,29". Hope this helps you, Matt
... View more
06-07-2021
07:50 AM
@myuintelli2021 Assisting on your new post here: https://community.cloudera.com/t5/Support-Questions/Nifi-untrusted-proxy-caused-by-Untrusted-Proxy-Exception/m-p/317796/highlight/false#M227327 Your choice of user authentication does not matter here. Authentication and Authorization processes are handled independently of one another. The Authentication of users/clients results in a string which is evaluated against identity mapping properties and then passed to the configured authorizer for authorization. Your exception points and missing /proxy authorization for your node strings. Hope this helps, Matt
... View more