Member since
07-30-2019
2757
Posts
1366
Kudos Received
798
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
324 | 11-30-2023 07:00 AM | |
147 | 11-02-2023 10:20 AM | |
320 | 11-02-2023 09:54 AM | |
337 | 11-01-2023 06:17 AM | |
255 | 10-31-2023 06:30 AM |
12-07-2023
08:54 AM
Hello , I have same issue and i tried the recommended solution but none of this works the same error still persists. I tried setting in /etc/hosts file my IP addresses as different names like my.nifi.server , my.server.com etc.. and set that in the nifi.properties file yet no difference in the error. I restarted several times and log error persists.
... View more
12-07-2023
12:27 AM
@Coordinador, have any of the replies helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
12-06-2023
07:03 AM
Hi @Fayza , You need to setup the Content-Type as required by the API. There should be "Request Content-Type" property where you can set the value. Also any custom header values can be added as Dynamic Property. The invokehttp processor should be very flexible to accommodate the different API requirements and request types.
... View more
12-01-2023
01:13 PM
You are awesome @SAMSAL Thanks so much for the great information. This helps me a lot. Thank goodness for this community, because I think the documentation is nearly adequate.
... View more
12-01-2023
10:45 AM
@Fanxxx Do you have a load balancer in front of your NiFi Cluster? If so, the load balancer needs to be configured to use session persistence (also known as sticky sessions). That is because when you access node 1 for example and authenticate your user that token is only valid for that specific node (there is a client token and a matching server side key). If subsequent request/redirect is sent to a different node by the load balancer the node is going to be missing the matching server side key and reject the client token. Sharing more details about your NiFi security setup and Apache NiFi version is also valuable to those who may provide suggestion in the community. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
09:24 AM
1 Kudo
@Zifo1 Welcome to the community.... The Single User Authorizer is not a full featured authorization provider. It was only added to Apache NiFi so that out-of-the-box NiFi would be able to start securely easily for evaluation purposes. It does not provide a mechanism for creating additional authorizations for other identities such as other NiFi instances. In order to support authorizing additional user/client identities against various NiFi policies, you'll need to switch to using a production ready authorizer like the "managed-authorizer". A typical example configuration would look like this: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-kerberos-authentication This setup uses the File-user-group-provider and file-access-policy-provider with the managed -authorizer. Now you could configure your single user provider identity as the "Initial User Identity 1" in the file-user-group-provider and as the "initial admin identity" in the file-access-policy-provider. This would setup the needed admin policies for this user identity you need. Note: Keep in mind that these providers will only generate the authorizations.xml and users.xml files the first time NiFi is started with this configuration. So if you set the above initial user and initial admin identities wrong, you'll need to fix config, delete these two files and start NiFi again so they are created again. Above does nothing with authentication since you are still using the single user authentication. With this default authentication provider you can only authenticate with the single user identity or using a clientAuth certificates (which may also be challenging with default truststore your out-of the-box NiFi uses). Authentication via a mutualTLS exchange is how Nifi node to node communications work and NiFi site-to-Site. In order for mutual TLS exchange to be successful there must be mutual trust of the certificate exchanged. So if one NiFi's certificate is not trusted by the other NiFi's truststore, authentication will not be possible. So you may need to add additional trustedCertEntries (public certs) to both your NiFi's truststores before you'll be able to successfully negotiate the MutualTLS exchange/handshake. All the available authentication providers offered are documented here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Aside from the single-user authentication provider, all other providers rely on some external source. Apache NiFi does not offer a multi-user local authentication provider. I know this is a lot of info thus far, but should provide you the path to a slightly more production ready NiFi that will open up the ability to use additional features not available with the out-of-the-box setup. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
08:57 AM
@Jisson I don't see ExecuteStream in the thread dump provided. Let's clarify first what you mean by "stuck"... When the processor is in this "stuck" state, does the processor indicate that it has an active thread? A NiFi processor will show a small number in its upper right corner when it has an active thread(s). Below example shows an ExecuteStreamCommand processor with "1" active thread: If your processor has no active threads, it is not stuck/hung. It is simply does not have a thread to execute the command. This could happen if all thread from the max timer driven thread pool in NiFi are already being used by other components. We would call this a thread starved processor. If your CPU load average is good, you could increase the size of the thread pool to see if that helps. NiFi out-of-the-box sets the "Maximum Timer Driven Thread Count" Pool to 10. You can change this from the NiFi Ui --> global menu (upper right corner) --> Controller Settings --> General tab. If your processor does show an active threads, i'd expect to see that thread in the thread dump. Also keep in mind that a single thread dump is not very useful. A thread may not be HUNG, but rather long running for example. So getting a series of thread dumps spread out to compare would allow you to see if the thread stack is changing over time indicating not hung but slow. In the case of your ExecuteStreamCommand processor, it is calling a custom python script and the waits for the return from that script. Then comes the challenge is the thread dump indicates it is waiting on your python script return to figure out why your python scripts is hanging or taking a very long time all of a sudden. Not something that can be troubleshot through NiFi. Hope this helps you in your troubleshooting journey. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
08:21 AM
@SAMSAL The managed Authorizer uses the file-access-policy-provider (generates the authorizations.xml if it does no already exist) and then a user-group-provider. In your case that would make most sense to be the ldap-user-group-provider. You may also want to use the Composite-configurable-user-group-provider (configure it with ldap-user-group-provider and file-user-group-provider). Having both a file based provider and ldap provider allows sycning of ldap users and groups form ldap automatically as well as the file provider allowing you to manually add non ldap user/client identities for authorization as well. Non ldap client/user identities might be certifcate based clients like other NiFi nodes/instance, etc.. Within the file-access-policy-provider you define the initial admin identity. That user identity could be set to your ldap user account identity. Then on first start up with managed provider, it generates the authorizations.xml file seeded with the policies necessary for that initial admin user identity to act as admin. So you could skip the single-user-provider step. Matt
... View more
12-01-2023
08:10 AM
@edtech And the ListenSyslog and putSyslog processors supports UDP: If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
12-01-2023
07:13 AM
@hegdemahendra You also need to be careful with some rest-api request as they may generate reports that are held in NiFi heap until a rest-api call is made to remove them. Doing similar actions directly from the UI handles the multiple calls needed automatically. So calls like these can slowly eat away at the NiFi heap impacting performance until a restart. Provenance queries would be an example of this. Unrelated note: NiFi Variable registry has been deprecated in favor of NiFi Parameters now. The Variable registry functionality has been officially removed starting with the latest NiFi 2.0.0-M1 release. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more