Member since
07-30-2019
3470
Posts
1642
Kudos Received
1018
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 275 | 05-06-2026 09:16 AM | |
| 457 | 05-04-2026 05:20 AM | |
| 335 | 05-01-2026 10:15 AM | |
| 516 | 03-23-2026 05:44 AM | |
| 390 | 02-18-2026 09:59 AM |
04-10-2023
11:43 AM
@niclyx While NiFi only supports TLS 1.2 inbound connections, it will support lower TLS version for outgoing. The TLS version used is negotiated between client (NIFi) and server end point during the TSL exchange. The highest level supported by both client and server is what will end up being used. @apmmahesh Authentication and Authorization are two separate process in NiFi. Successful Authentication will result in an user identity string (case sensitive) which is then passed to the authorization process to verify if the that identity string has been authorized for the requested resource. In order to access the NiFI UI, users must be authorized to "view the user interface". The authorization is configured in the authorizers.xml. I am not sure which authorizer or providers you are using. Most commonly used is the managed authorizer, File-access-policy-provider, and File-user-group-provider. But you might also be using the ldap-user-group-provider. The file-access-policy-provider is used to setup necessary policies for the initial admin and nodes (if multi-node NiFi cluster) in an authorizations.xml file. Within the file-access-policy provider you would define your initial admin user identity string "xyz123" so that the polcies need to be admin are assigned to this user identity. Keep in mind the file-access-policy provider will only generate an authorizations.xml file if it does NOT already exist. So if you edit the initial admin or NiFi nodes later, this changes willl not be reflected in the already existing authorizations.xml. You'll need to delete current so new is generated. Before the file access-policy-provider can setup initial admin policies for user "xyz123", the authorizer needs to be able to return that user identity (case sensitive) from one of the configured user-group-provider configured on the authorizers.xml. You can use the file-user-group-provider to manually add user identities to NiFi for purpose of setting up aithorization (in this case their is an initial user identity 1 property you would add "xyz123" to. Optionally you can setup the ldap-user-group-provider to sync user and group identities from your ldap server. Keep in mind that two providers can not return the same identity. So if you are syncing from LDAP, don't configure the same user identity in the file-user-group-provider. here are the relevant Apache NiFi docs on these providers: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileusergroupprovider https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldapusergroupprovider https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#composite-implementations https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileaccesspolicyprovider https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#standardmanagedauthorizer If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-10-2023
11:40 AM
1 Kudo
@apmmahesh Authentication and Authorization are two separate process in NiFi. Successful Authentication will result in an user identity string (case sensitive) which is then passed to the authorization process to verify if the that identity string has been authorized for the requested resource. In order to access the NiFI UI, users must be authorized to "view the user interface". The authorization is configured in the authorizers.xml. I am not sure which authorizer or providers you are using. Most commonly used is the managed authorizer, File-access-policy-provider, and File-user-group-provider. But you might also be using the ldap-user-group-provider. The file-access-policy-provider is used to setup necessary policies for the initial admin and nodes (if multi-node NiFi cluster) in an authorizations.xml file. Within the file-access-policy provider you would define your initial admin user identity string "xyz123" so that the polcies need to be admin are assigned to this user identity. Keep in mind the file-access-policy provider will only generate an authorizations.xml file if it does NOT already exist. So if you edit the initial admin or NiFi nodes later, this changes willl not be reflected in the already existing authorizations.xml. You'll need to delete current so new is generated. Before the file access-policy-provider can setup initial admin policies for user "xyz123", the authorizer needs to be able to return that user identity (case sensitive) from one of the configured user-group-provider configured on the authorizers.xml. You can use the file-user-group-provider to manually add user identities to NiFi for purpose of setting up aithorization (in this case their is an initial user identity 1 property you would add "xyz123" to. Optionally you can setup the ldap-user-group-provider to sync user and group identities from your ldap server. Keep in mind that two providers can not return the same identity. So if you are syncing from LDAP, don't configure the same user identity in the file-user-group-provider. here are the relevant Apache NiFi docs on these providers: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileusergroupprovider https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldapusergroupprovider https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#composite-implementations https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileaccesspolicyprovider https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#standardmanagedauthorizer If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-07-2023
07:59 AM
@axelfulop NiFi Authorization is case sensitive. In your authorizers.xml (responsible for generating the users.xml and authorizationsxml files if and only if they do NOT already exist) you have defined your "initial user identity 1" and "Iniital admin identity" as: axel.fulop@paycomplete.com However, your attached screenshot of the the NiFi UI, shows the authenticated user's 'user identity" was: Axel.Fulop@paycomplete.com NiFi's authorizer will treat these two identities as different user identities since it is case sensitive. There are two ways to resolve this: 1. Update Authorizers.xml entries to use "Axel.Fulop@paycomplete.com", remove users.xml, remove authorizations.xml and then restart NiFi. These two files will get recreated based on new user identity. 2. Add a set of identity mapping properties to the nifi.properties file that will take all authenticated user identities and set them to all lowercase: nifi.security.identity.mapping.pattern.username=^(.*?)$
nifi.security.identity.mapping.value.username=$1
nifi.security.identity.mapping.transform.username=LOWER This would result in the identity string resulting from successful authentication is converted to all lowercase before being passed to the configured NiFi authorizer. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-06-2023
12:19 PM
@Meeran Did you setup sticky sessions in your load balancer? When you provide your ldap username and password, a JWT Token issue to the client which is only valid for use with the specific NiFi node that issued it. So if you enter your username and password and you LB sends next request to a different node to load the flow, the JWT token will not be accepted to identify your user on that other node. Watch the nifi-user.log on all nodes when you attempt to login. In one of those logs you will see authentication success logged for your user. I suspect that then the above exception about "anonymous" is then in a different node's user log. If you are having trouble configuring your load balancer, it would be best to start a new community question for that query so it gets to a wider audience. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-06-2023
11:55 AM
@drewski7 I have not been able to reproduce the NiFi Registry URL truncation yet using Apache NiFi 1.19 I have installed. Couple things to check/try: 1. Apache NiFi 1.16.0 is about the time NiFi switched to using the flow.json.gz to load the flow in to memory at startup. Every time you make a change via the NiFi Ui, the current flow.json.gz is archived and new flow.json.gz is created. After fixing your NiFi-Registry URL, go inspect the flow.json.hgz file and see if the URL is being truncated when being persisted out to the flow.json.gz file. This file is only read at startup to load the flow back into NiFi's heap memory. The Registry clients will be at very beginning of the flow.json.gz 2. Try adding the port to the URL and see if that makes a difference (https://xyz.com:443/gateway/registry/nifi-registry-app/nifi-registry-api ). If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-06-2023
06:26 AM
@drewski7 Why are you connecting your NiFi to NiFi-Registry via Knox instead of directly[y connecting to the NiFi-registry host? NiFi will authenticate via a mutual TLS exchange using the keystore and truststores configured in nifi.properties and the nifi-registry.properties files. Thanks, Matt
... View more
04-05-2023
11:46 AM
1 Kudo
@RRosa This is likely result of changes on Twitter's side related to changes to their twitter API endpoint and not directly related to the ConsumeTwitter processor included with NiFi: https://techcrunch.com/2023/02/01/twitter-to-end-free-access-to-its-api/ Looks like free access to consume from Twitter was cut off in February. From the following page, we can see that the Free tier now only allows writes and no consumes. https://developer.twitter.com/en/products/twitter-api You would need at least a "Basic" subscription to continue to consume from Twitter. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-05-2023
11:30 AM
@davehkd Keep in mind that you are not adding users to either NiFi or NiFi-Registry that can be used for authentication. You are only adding "user identities" that match the user Identity that is returned post successful authentication. For example: Your current method of authentication is via a mutual TLS exchange. Once you pass your client certificate NiFi uses the DN from that certificate as the "user Identity" which is looked up locally to see which authorizations have been associated with that "user identity" (case sensitive). You can additional "user identities" manually right now via the NiFi and NiFi-Registry UI. Then you can assign authorizations to those newly added "user identities". You can use the NiFi TLS toolkit to also create a certificate for each additional user to use for authentication via the mutual TLS exchange. Many users of NiFi and NiFi-Registry use alternate methods of user authentication with "ldap-provider" as probably the most common. If you set up ldap-provider in the login-identity-providers.xml, you would be able to authenticate users via that ldap through a login window presented by NiFi instead of needing to use TLS certificates. You can then optionally, also add the ldap-user-group-provider to the configuration in the authorizers.xml to sync "user identities" and "group identities" from your ldap server so you do not need to manually add these identities yourself via the UI. Hope this clarifies some things for you. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-04-2023
01:20 PM
@Meeran Where are you seeing above exception? System Error
The request contained an invalid host header [internal-myappname-13718787520.us-east-1.elb.amazonaws.com:8443] in the request [/]. Check for request manipulation or third-party intercept. In your Load Balancer logs? This host header "internal-myappname-13718787520.us-east-1.elb.amazonaws.com" is in your nifi.web.proxy.host list or in the SAN list within each of your node's certificates? In NiFi, check the nifi-user.log and nifi-app.log for any WARN or ERROR log output being produced when you try to access NiFi via the load balancer URL.
... View more
04-04-2023
12:41 PM
1 Kudo
@Dave0x1 Apache NiFi takes carful consideration when upgrading/migrating from an older version to a newer version with in reason. Like upgrading from 1.13 to 1.19 would be expected to pose no real issues; however, jumping from 1.10 directly to 1.19 may pose some challenges. That being said, rolling back is not something that can be guaranteed to go smoothly as some versions can result in changes to database structure/content, modification/change to NiFi components (processor, controller services, reporting tasks, etc). In a best case scenario, a rollback my result in a working NiFi, with unexpected unexpected invalid components on the NiFi canvas. This happens when new properties are introduced in components being used, Those component changes get written to the flow.xml.gz and new flow.josn.gz files. So on rollback those new properties turn in to dynamic properties which may result in an invalid component until they are removed manually. In other scenarios, there may have been upgrades/changes to H2 DB that won't rollback smoothly. So perhaps taking a backup of the flow.xml.gz. flow.json.gz, and database_repository/ directory contents would be useful in the vent of rollback. Good news is that there is nothing in the H2 DBs located in the database_repository you need to retain. The nifi-flow-audit DB tracks history of changes you made on the canvas. The other user and identity dbs can just be removed and allowed to be recreated on next startup. So in your rollback test, I would rename the existing DB files and restart allowing new to be created. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more