Member since
07-30-2019
3472
Posts
1642
Kudos Received
1020
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 250 | 06-03-2026 06:06 PM | |
| 527 | 05-06-2026 09:16 AM | |
| 1027 | 05-04-2026 05:20 AM | |
| 584 | 05-01-2026 10:15 AM | |
| 699 | 03-23-2026 05:44 AM |
04-17-2023
01:27 PM
@davehkd Thank you for the awesome feedback. Glad i could help you. It would be great if you could go through this back and forth exchange and accept all the responses I provided that helped you. I noticed you accepted your own response. 🙂 Matt
... View more
04-17-2023
01:24 PM
@niclyx I think you are diverging form the original query asked in this community question which has an accepted solution. When it comes to a mutual TLS exchange/handshake, this is not something NiFi created or has made any modification to. There are many different resources on the web to learn about MutualTLS and how it works. NiFi only cares that the TLS handshake was successful and if no other method of authentication was setup, it received a clientAuth certificate from which it will use the full DN as the identity for the authenticated user. I recommend starting a new community questions if you have new questions. Matt
... View more
04-17-2023
01:03 PM
@Manika The single-user-provider used by NiFi out of the box does not support multiple users. It simply sets up a single user to access the secured NiFi suing the out of the box NiFi setup. Only the very first time NiFi is launched with this setup will the generated username and password be output in the nifi-app.log. You can use the following command to change that username and password: $ ./bin/nifi.sh set-single-user-credentials <username> <password> This single user username and password has full access to everything, so NiFi does not expose "policies" as setting up additional users is not possible. NiFi provides no other provider for local users. NiFi historically has left user management as an external dependency. In order tp support multiple users with and various level of authorization, you will need to switch to using a different authorizer and authentication method. Switching to another authentication method will require you establish an external means for user authentication. NiFi has numerous supported methods (User certificate, LDAP, Kerberos, etc.) https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Once you have selected the method you will use to authenticate your users that is not the "single-user-provider", you need to change your authorizer from "single-user-authorizer" to "standard-managed-authorizer": https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#standardmanagedauthorizer If you found that any of the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
10:08 AM
@davehkd 1. You can't have multiple providers in the authorizers.xml. You'll ned to comment out the "singleUserAuthorizer". 2. Your "managed authorizer is not correct, you are missing the class: <authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer> 3. Your file-access-policy-providers configuration looks good here. 4. Your file-user-group-provider configuration is not correct, you are missing the line that defines where the users.xml file is to be created: <userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Initial User Identity 1">CN=ec2-user</property>
<property name="Initial User Identity 2">CN=nifi1, OU=NIFI</property>
<property name="Initial User Identity 3">CN=nifi2, OU=NIFI</property>
<property name="Initial User Identity 4">CN=nifi3, OU=NIFI</property>
</userGroupProvider> Make sure that in your nifi.properties file, you have configured NiFi to use the managed authorizer: nifi.security.user.authorizer=managed-authorizer Make sure that in your nifi.properties, you do NOT have the following property configured to use the single-user-provider: nifi.security.user.login.identity.provider= If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
09:58 AM
@davehkd The "Initial Admin Identity" is defined in the file-access-policy provider in the authorizers.xml. This provider is executed during startup of the application startup and creates an authorizations.xml file where it we set a pre-defined set of NiFi Resource Identifiers (NiFi policies) needed for NiFi administration by the Initial Admin Identity and set NiFi Resource Identifiers needed by the NiFI nodes defined in "Node Identity <num>". The file-access-policy-provider has a dependency on a user-group-provider. This dependency exists because NiFi Resource Identifiers (NiFi Policies) can't be set for any user/client identity strings that are not provided by one of the user-group-providers NiFi offers. In your case, it appears you are using the file-user-group-provider, which means the nodes and initial admin must be added as "Initial user identity <num>" in that provider. The file-user-group-provider is responsible for creating the users.xml. IMPORTANT: These providers will only generate a users.xml and authorizations.xml file if they do NOT already exist. Expectation is that once created, additional users and setting of additional policies occurs from within the UI. So configuration changes to these providers will have no impact on already existing users.xmll and authorizations.xml. So if mistakes where made in configuration resulting in missing needed content in these files, they need to removed after making your config changes so new can be generated. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
09:04 AM
@davehkd That is an incomplete authorizers.xml and thus not valid. Matt
... View more
04-12-2023
08:58 AM
@apmmahesh I see one issue with your shared authorizers.xml configuration. The file loads the various providers from the top down, so it is VERY important that if a provider has a dependency on another provider that the provider it is dependent on is loaded first. So in your authorizers.xml has the following loading order: file-access-policy-provider --> file-user-grouo-provider --> managed-authorizer The problem here is that the file-access-policy provider is dependent on the file-user-group-provider which has not been loaded yet. so it will not set default policies correctly when generating the authorizations.xml file. Move the file-access-policy-provider between fie-user-group-provider (top of authorizers.xml) and managed-authorizer (bottom of authorizers.xml). Second issue is that you have properties "Initial Admin Identity 1" and "Initial Admin Identity 2" properties in your file-access-policy-provider. These are not real properties. You can only specify one "Initial Admin Identity" in the file-access-policy-provider. These unexpected property names would just be ignored. Third issue is that you need to not only specify your "Initial Admin Identity" in the file-access-policy-provider, but also need to make sure that the same user identity string is being defined in the "file-user-group-provider as another "initial User Identity <num>". NiFi can't seed policies for a user Identity it does know about. Also keep in mind that changes to authorizers.xml file-access-policy-provider and file-access-policy-provider configuration will not result in modification of existing users.xml and authorizations.xml files. You'll need to remove these so that new are created on startup of NiFi. Now if you can already login with your "CN=Admin, OU=NiFi" certificate and at time of that setup the users.xml and authorizations.xml was created correctly, you can use this admin user to access "users" from global menu in NiFi and add your additional user "xyz123" and then set policies you want user xyz123 to have. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
07:52 AM
2 Kudos
@drewski7 The removal of quotes from the "command arguments" is expected behavior in the ExecuteStreamCommand processor. This processor was introduced to NiFi more than 10 years ago and was originally designed for a more minimal scope of work including the expectation that FlowFile content would be passed to the script/command being executed. As time passed on the use cases that were trying to be solutioned via the ExecuteStreamCommand expanded; however, handling those use case would potential break users already implemented and working dataflow. So rather then change that default behavior, a new property "Command Arguments Strategy" was added with the original "Command Arguments Property" as the default (legacy method) and a new "Dynamic property arguments" option. This change is part of this JIra and implemented as of Apache NiFi 1.10: https://issues.apache.org/jira/browse/NIFI-3221 In your use case, you'll want to switch to using the "Dynamic property arguments". This will then require you to click on the "+" to add a new dynamic property. The property names MUST use this format: command.argument.<num> So in your case you might try something like: command.argument.1 = -X POST -H referer:${Referer} -H 'Content-Type: application/json' -d '{"newTopics": [{"name":"testing123","numPartitions":3,"replicationFactor":3}], "allTopicNames":["testing123"]}' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/topics If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
07:22 AM
@davehkd It is difficult for me t say where your configuration has gone wrong with what has been shared I would need to see your authorizers.xml. I see in your output mention of the SingleUserAuthorizer. This authorizer does not support defining additional user for assigning policies. What do you have configured in your nifi.properties for these configuration lines: nifi.security.user.authorizer
nifi.security.user.login.identity.provider If either of these is using the "single-user-provider/authorizer", it is expected that you will not see users in the UI. You'll need to use an authorizer that supports multi user setup like the https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#standardmanagedauthorizer. since you are using user certificates for handling your user authentication. the nifi.security.user.login.identity.provider configuration property should be blank. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
07:09 AM
1 Kudo
@cotopaul Your issue is caused by trying to use the ":toDate" and ":format" functions here. These two functions utilize the Java SimpleDateFormat which can not handle your date which uses nanoseconds. You'll want to use a NiFi Expression Language function that supports the Java DateTimeFormatter like ":formatInstant", "toInstant", ":toNanos", or ":toMicros". Using your example: sql.args.2.value = 2023-04-10T10:43:15.794241429+03:00 and an updateAttribute processor configured with this new dynamic property: newformat = ${sql.args.2.value:formatInstant('yyyy-MM-dd HH:mm:ss.SSS','GMT')} Will result in the newformat FlowFile attribute with a value of: 2023-04-10 07:43:15.794 You'll notice the output was designated as being GMT so 3 hours were removed since you had a +03:00 GMT/UTC offset in your original time format. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more