Member since
07-30-2019
3470
Posts
1642
Kudos Received
1018
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 275 | 05-06-2026 09:16 AM | |
| 457 | 05-04-2026 05:20 AM | |
| 335 | 05-01-2026 10:15 AM | |
| 516 | 03-23-2026 05:44 AM | |
| 390 | 02-18-2026 09:59 AM |
04-12-2023
10:08 AM
@davehkd 1. You can't have multiple providers in the authorizers.xml. You'll ned to comment out the "singleUserAuthorizer". 2. Your "managed authorizer is not correct, you are missing the class: <authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer> 3. Your file-access-policy-providers configuration looks good here. 4. Your file-user-group-provider configuration is not correct, you are missing the line that defines where the users.xml file is to be created: <userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Initial User Identity 1">CN=ec2-user</property>
<property name="Initial User Identity 2">CN=nifi1, OU=NIFI</property>
<property name="Initial User Identity 3">CN=nifi2, OU=NIFI</property>
<property name="Initial User Identity 4">CN=nifi3, OU=NIFI</property>
</userGroupProvider> Make sure that in your nifi.properties file, you have configured NiFi to use the managed authorizer: nifi.security.user.authorizer=managed-authorizer Make sure that in your nifi.properties, you do NOT have the following property configured to use the single-user-provider: nifi.security.user.login.identity.provider= If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
09:58 AM
@davehkd The "Initial Admin Identity" is defined in the file-access-policy provider in the authorizers.xml. This provider is executed during startup of the application startup and creates an authorizations.xml file where it we set a pre-defined set of NiFi Resource Identifiers (NiFi policies) needed for NiFi administration by the Initial Admin Identity and set NiFi Resource Identifiers needed by the NiFI nodes defined in "Node Identity <num>". The file-access-policy-provider has a dependency on a user-group-provider. This dependency exists because NiFi Resource Identifiers (NiFi Policies) can't be set for any user/client identity strings that are not provided by one of the user-group-providers NiFi offers. In your case, it appears you are using the file-user-group-provider, which means the nodes and initial admin must be added as "Initial user identity <num>" in that provider. The file-user-group-provider is responsible for creating the users.xml. IMPORTANT: These providers will only generate a users.xml and authorizations.xml file if they do NOT already exist. Expectation is that once created, additional users and setting of additional policies occurs from within the UI. So configuration changes to these providers will have no impact on already existing users.xmll and authorizations.xml. So if mistakes where made in configuration resulting in missing needed content in these files, they need to removed after making your config changes so new can be generated. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
09:04 AM
@davehkd That is an incomplete authorizers.xml and thus not valid. Matt
... View more
04-12-2023
08:58 AM
@apmmahesh I see one issue with your shared authorizers.xml configuration. The file loads the various providers from the top down, so it is VERY important that if a provider has a dependency on another provider that the provider it is dependent on is loaded first. So in your authorizers.xml has the following loading order: file-access-policy-provider --> file-user-grouo-provider --> managed-authorizer The problem here is that the file-access-policy provider is dependent on the file-user-group-provider which has not been loaded yet. so it will not set default policies correctly when generating the authorizations.xml file. Move the file-access-policy-provider between fie-user-group-provider (top of authorizers.xml) and managed-authorizer (bottom of authorizers.xml). Second issue is that you have properties "Initial Admin Identity 1" and "Initial Admin Identity 2" properties in your file-access-policy-provider. These are not real properties. You can only specify one "Initial Admin Identity" in the file-access-policy-provider. These unexpected property names would just be ignored. Third issue is that you need to not only specify your "Initial Admin Identity" in the file-access-policy-provider, but also need to make sure that the same user identity string is being defined in the "file-user-group-provider as another "initial User Identity <num>". NiFi can't seed policies for a user Identity it does know about. Also keep in mind that changes to authorizers.xml file-access-policy-provider and file-access-policy-provider configuration will not result in modification of existing users.xml and authorizations.xml files. You'll need to remove these so that new are created on startup of NiFi. Now if you can already login with your "CN=Admin, OU=NiFi" certificate and at time of that setup the users.xml and authorizations.xml was created correctly, you can use this admin user to access "users" from global menu in NiFi and add your additional user "xyz123" and then set policies you want user xyz123 to have. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
07:52 AM
2 Kudos
@drewski7 The removal of quotes from the "command arguments" is expected behavior in the ExecuteStreamCommand processor. This processor was introduced to NiFi more than 10 years ago and was originally designed for a more minimal scope of work including the expectation that FlowFile content would be passed to the script/command being executed. As time passed on the use cases that were trying to be solutioned via the ExecuteStreamCommand expanded; however, handling those use case would potential break users already implemented and working dataflow. So rather then change that default behavior, a new property "Command Arguments Strategy" was added with the original "Command Arguments Property" as the default (legacy method) and a new "Dynamic property arguments" option. This change is part of this JIra and implemented as of Apache NiFi 1.10: https://issues.apache.org/jira/browse/NIFI-3221 In your use case, you'll want to switch to using the "Dynamic property arguments". This will then require you to click on the "+" to add a new dynamic property. The property names MUST use this format: command.argument.<num> So in your case you might try something like: command.argument.1 = -X POST -H referer:${Referer} -H 'Content-Type: application/json' -d '{"newTopics": [{"name":"testing123","numPartitions":3,"replicationFactor":3}], "allTopicNames":["testing123"]}' --negotiate -u : -b /tmp/cookiejar.txt -c /tmp/cookiejar.txt http://SMM-HOSTNAME:8585/api/v1/admin/topics If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
07:22 AM
@davehkd It is difficult for me t say where your configuration has gone wrong with what has been shared I would need to see your authorizers.xml. I see in your output mention of the SingleUserAuthorizer. This authorizer does not support defining additional user for assigning policies. What do you have configured in your nifi.properties for these configuration lines: nifi.security.user.authorizer
nifi.security.user.login.identity.provider If either of these is using the "single-user-provider/authorizer", it is expected that you will not see users in the UI. You'll need to use an authorizer that supports multi user setup like the https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#standardmanagedauthorizer. since you are using user certificates for handling your user authentication. the nifi.security.user.login.identity.provider configuration property should be blank. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
07:09 AM
1 Kudo
@cotopaul Your issue is caused by trying to use the ":toDate" and ":format" functions here. These two functions utilize the Java SimpleDateFormat which can not handle your date which uses nanoseconds. You'll want to use a NiFi Expression Language function that supports the Java DateTimeFormatter like ":formatInstant", "toInstant", ":toNanos", or ":toMicros". Using your example: sql.args.2.value = 2023-04-10T10:43:15.794241429+03:00 and an updateAttribute processor configured with this new dynamic property: newformat = ${sql.args.2.value:formatInstant('yyyy-MM-dd HH:mm:ss.SSS','GMT')} Will result in the newformat FlowFile attribute with a value of: 2023-04-10 07:43:15.794 You'll notice the output was designated as being GMT so 3 hours were removed since you had a +03:00 GMT/UTC offset in your original time format. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-12-2023
05:27 AM
@linssab What version of Apache NiFi are you using and where did you download it from? The Single-user-provider and single-user-authorizer where introduced to NiFi starring with Apache NiFi 1.16. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-10-2023
01:49 PM
1 Kudo
@davehkd Your corporate PKI certs are just certs signed by your corporate signing authority. So really nothing needs to change in the configuration of your NiFi. However, you will need to add your corporate signing authorities public certificate to the truststore.jks used by your NiFi. When you access the NiFi UI a mutual TLS exchange occurs. You client (browser) initiates the connection with a client Hello to which the server will respond with a server Hello. In that server hello a list of certificate authorities (comes from CAs in the NiFi truststore) will be provided to back to the client. This tells the client that it will only accept a client certificate signed by one of those authorities. If the client (your browser) has a pki certificate loaded that was signed by one of those trusted authorities, that client certificate can be used to complete the mutual TLS exchange. Much more happens in the Mutual TLS client server handshake, but i'll just stick to the certificate specifics here. NiFi does nothing out of spec when it comes to certificates and NiFi has nothing to do with creating users. You can create your own self signed certificates created via command line tools, use a public signing authority to create signed certificates, or in your case use your corporate signing authority to create certificates. A java keystore is nothing more than a container that can hold 1 too many certificates. Only requirement NiFi has with regards to the keystore it that it contains only 1 PrivateKeyEntry (private cert). The truststore (just another keystore) can contain 1 too many trustedCertEntries (public certs) and often contains many public intermediate and root CAs. If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more
04-10-2023
01:33 PM
@ahmedalsaidi You do not need to change the content type since you specify the character set to use in the ExtractText processor which defaults to "UTF-8". If you change the content type or the filename, the built in content viewer in NiFi would be able to display text instead of hex. For example: adding ".txt" to end of filename. When it comes to your matching issue, it would be difficult for me to say what is happening here without a working and non-working sample to look at. In Java regular expressions the "." means any character; however, looking at you hex output screenshot it looks like you really want a literal "." to match. If you want your java regular expression to match the literal ".", then add a "\" (backslash) before each ".". If you found that the provided solution(s) assisted you with your query, please take a moment to login and click Accept as Solution below each response that helped. Thank you, Matt
... View more