Member since
07-30-2019
2757
Posts
1366
Kudos Received
798
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
370 | 11-30-2023 07:00 AM | |
157 | 11-02-2023 10:20 AM | |
325 | 11-02-2023 09:54 AM | |
341 | 11-01-2023 06:17 AM | |
258 | 10-31-2023 06:30 AM |
12-01-2023
11:47 AM
1 Kudo
@ChuckE You can right click on the UpdateAttribute processor and "view state" which will allow you to clear the stored state for that processor without needing to create a new processor. Every dynamic property you add becomes a different local state variable that is added to the FlowFiles as a FlowFile Attribute. Thanks, Matt
... View more
12-01-2023
11:37 AM
2 Kudos
@Fayza Apache CouchDB looks very limited in its query capabilities. But it does appear that a user in the community created nifi-couchDB components: https://github.com/wasperen/nifi-couchdb You could try adding this nar to your Apache NiFi installation to see if these components can satisfy your use case. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
10:45 AM
@Fanxxx Do you have a load balancer in front of your NiFi Cluster? If so, the load balancer needs to be configured to use session persistence (also known as sticky sessions). That is because when you access node 1 for example and authenticate your user that token is only valid for that specific node (there is a client token and a matching server side key). If subsequent request/redirect is sent to a different node by the load balancer the node is going to be missing the matching server side key and reject the client token. Sharing more details about your NiFi security setup and Apache NiFi version is also valuable to those who may provide suggestion in the community. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
10:37 AM
@whoknows Apache NiFi 2.0.0-M1 requires Java 21 and utilizes Jetty 10. This results in needing to comply with the SNI specification. So the URL used to access your NiFi can not use an IP address and the hostname used must match a hostname found in the SAN entries list with the NiFi's configured keystore PrivateKeyEntry. Apache NiFi 2.0 out-of-the-box will generate a keystore and truststore. The keystore will contain a PrivateKeyEntry with a SAN entry for localhost and the server hostname. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
09:24 AM
1 Kudo
@Zifo1 Welcome to the community.... The Single User Authorizer is not a full featured authorization provider. It was only added to Apache NiFi so that out-of-the-box NiFi would be able to start securely easily for evaluation purposes. It does not provide a mechanism for creating additional authorizations for other identities such as other NiFi instances. In order to support authorizing additional user/client identities against various NiFi policies, you'll need to switch to using a production ready authorizer like the "managed-authorizer". A typical example configuration would look like this: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-kerberos-authentication This setup uses the File-user-group-provider and file-access-policy-provider with the managed -authorizer. Now you could configure your single user provider identity as the "Initial User Identity 1" in the file-user-group-provider and as the "initial admin identity" in the file-access-policy-provider. This would setup the needed admin policies for this user identity you need. Note: Keep in mind that these providers will only generate the authorizations.xml and users.xml files the first time NiFi is started with this configuration. So if you set the above initial user and initial admin identities wrong, you'll need to fix config, delete these two files and start NiFi again so they are created again. Above does nothing with authentication since you are still using the single user authentication. With this default authentication provider you can only authenticate with the single user identity or using a clientAuth certificates (which may also be challenging with default truststore your out-of the-box NiFi uses). Authentication via a mutualTLS exchange is how Nifi node to node communications work and NiFi site-to-Site. In order for mutual TLS exchange to be successful there must be mutual trust of the certificate exchanged. So if one NiFi's certificate is not trusted by the other NiFi's truststore, authentication will not be possible. So you may need to add additional trustedCertEntries (public certs) to both your NiFi's truststores before you'll be able to successfully negotiate the MutualTLS exchange/handshake. All the available authentication providers offered are documented here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Aside from the single-user authentication provider, all other providers rely on some external source. Apache NiFi does not offer a multi-user local authentication provider. I know this is a lot of info thus far, but should provide you the path to a slightly more production ready NiFi that will open up the ability to use additional features not available with the out-of-the-box setup. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
08:57 AM
@Jisson I don't see ExecuteStream in the thread dump provided. Let's clarify first what you mean by "stuck"... When the processor is in this "stuck" state, does the processor indicate that it has an active thread? A NiFi processor will show a small number in its upper right corner when it has an active thread(s). Below example shows an ExecuteStreamCommand processor with "1" active thread: If your processor has no active threads, it is not stuck/hung. It is simply does not have a thread to execute the command. This could happen if all thread from the max timer driven thread pool in NiFi are already being used by other components. We would call this a thread starved processor. If your CPU load average is good, you could increase the size of the thread pool to see if that helps. NiFi out-of-the-box sets the "Maximum Timer Driven Thread Count" Pool to 10. You can change this from the NiFi Ui --> global menu (upper right corner) --> Controller Settings --> General tab. If your processor does show an active threads, i'd expect to see that thread in the thread dump. Also keep in mind that a single thread dump is not very useful. A thread may not be HUNG, but rather long running for example. So getting a series of thread dumps spread out to compare would allow you to see if the thread stack is changing over time indicating not hung but slow. In the case of your ExecuteStreamCommand processor, it is calling a custom python script and the waits for the return from that script. Then comes the challenge is the thread dump indicates it is waiting on your python script return to figure out why your python scripts is hanging or taking a very long time all of a sudden. Not something that can be troubleshot through NiFi. Hope this helps you in your troubleshooting journey. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
08:33 AM
@Coordinador You could upgrade your MongoDB to a newer version. Another option you may try is to add the Apache NiFi 1.16 *mongo* nars to your Apache NiFi 1.22 custom extension folder: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#autoloading-processors This will make both the 1.16 and 1.22 versions of the mongo processor available for use in your 1.22 Apache NiFi. The Mongo Driver has been updated numerous times between 1.16 and 1.23: https://issues.apache.org/jira/browse/NIFI-9886 https://issues.apache.org/jira/browse/NIFI-10557 https://issues.apache.org/jira/browse/NIFI-11419 https://issues.apache.org/jira/browse/NIFI-11856 If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
08:21 AM
@SAMSAL The managed Authorizer uses the file-access-policy-provider (generates the authorizations.xml if it does no already exist) and then a user-group-provider. In your case that would make most sense to be the ldap-user-group-provider. You may also want to use the Composite-configurable-user-group-provider (configure it with ldap-user-group-provider and file-user-group-provider). Having both a file based provider and ldap provider allows sycning of ldap users and groups form ldap automatically as well as the file provider allowing you to manually add non ldap user/client identities for authorization as well. Non ldap client/user identities might be certifcate based clients like other NiFi nodes/instance, etc.. Within the file-access-policy-provider you define the initial admin identity. That user identity could be set to your ldap user account identity. Then on first start up with managed provider, it generates the authorizations.xml file seeded with the policies necessary for that initial admin user identity to act as admin. So you could skip the single-user-provider step. Matt
... View more
12-01-2023
08:10 AM
@edtech And the ListenSyslog and putSyslog processors supports UDP: If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
08:06 AM
2 Kudos
@ChuckE Plus 1 on @SAMSAL post. I also had no problem with it working on my CFM 2.1.5.3 SP3 (Based off Apache NiFi 1.18 with many bug fixes from 1.19 - 1.20) Thanks, Matt
... View more