@whoknows  Apache NiFi 2.0.0-M1 requires Java 21 and utilizes Jetty 10.  This results in needing to comply with the SNI specification.   So the URL used to access your NiFi can not use an IP address and the hostname used must match a hostname found in the SAN entries list with the NiFi's configured keystore PrivateKeyEntry. Apache NiFi 2.0 out-of-the-box will generate a keystore and truststore.  The keystore will contain a PrivateKeyEntry with a SAN entry for localhost and the server hostname. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Zifo1  Welcome to the community.... The Single User Authorizer is not a full featured authorization provider. It was only added to Apache NiFi so that out-of-the-box NiFi would be able to start securely easily for evaluation purposes.  It does not provide a mechanism for creating additional authorizations for other identities such as other NiFi instances. In order to support authorizing additional user/client identities against various NiFi policies, you'll need to switch to using a production ready authorizer like the "managed-authorizer". A typical example configuration would look like this: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-kerberos-authentication This setup uses the File-user-group-provider and file-access-policy-provider with the managed -authorizer. Now you could configure your single user provider identity as the "Initial User Identity 1" in the file-user-group-provider and as the "initial admin identity" in the file-access-policy-provider.  This would setup the needed admin policies for this user identity you need.   Note: Keep in mind that these providers will only generate the authorizations.xml and users.xml files the first time NiFi is started with this configuration. So if you set the above initial user and initial admin identities wrong, you'll need to fix config, delete these two files and start NiFi again so they are created again. Above does nothing with authentication since you are still using the single user authentication.  With this default authentication provider you can only authenticate with the single user identity or using a clientAuth certificates (which may also be challenging with default truststore your out-of the-box NiFi uses).   Authentication via a mutualTLS exchange is how Nifi node to node communications work and NiFi site-to-Site.  In order for mutual TLS exchange to be successful there must be mutual trust of the certificate exchanged.  So if one NiFi's certificate is not trusted by the other NiFi's truststore, authentication will not be possible.  So you may need to add additional trustedCertEntries (public certs) to both your NiFi's truststores before you'll be able to successfully negotiate the MutualTLS exchange/handshake. All the available authentication providers offered are documented here: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Aside from the single-user authentication provider, all other providers rely on some external source.  Apache NiFi does not offer a multi-user local authentication provider. I know this is a lot of info thus far, but should provide you the path to a slightly more production ready NiFi that will open up the ability to use additional features not available with the out-of-the-box setup. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Jisson  I don't see ExecuteStream in the thread dump provided.   Let's clarify first what you mean by "stuck"... When the processor is in this "stuck" state, does the processor indicate that it has an active thread?  A NiFi processor will show a small number in its upper right corner when it has an active thread(s).  Below example shows an ExecuteStreamCommand processor with "1" active thread: If your processor has no active threads, it is not stuck/hung.  It is simply does not have a thread to execute the command.  This could happen if all thread from the max timer driven thread pool in NiFi are already being used by other components.  We would call this a thread starved processor.  If your CPU load average is good, you could increase the size of the thread pool to see if that helps.  NiFi out-of-the-box sets the "Maximum Timer Driven Thread Count" Pool to 10.  You can change this from the NiFi Ui --> global menu (upper right corner) --> Controller Settings --> General tab. If your processor does show an active threads, i'd expect to see that thread in the thread dump.  Also keep in mind that a single thread dump is not very useful.  A thread may not be HUNG, but rather long running for example.  So getting a series of thread dumps spread out to compare would allow you to see if the thread stack is changing over time indicating not hung but slow.  In the case of your ExecuteStreamCommand processor, it is calling a custom python script and the waits for the return from that script.  Then comes the challenge is the thread dump indicates it is waiting on your python script return to figure out why your python scripts is hanging or taking a very long time all of a sudden. Not something that can be troubleshot through NiFi. Hope this helps you in your troubleshooting journey. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Coordinador  You could upgrade your MongoDB to a newer version. Another option you may try is to add the Apache NiFi 1.16 *mongo* nars to your Apache NiFi 1.22 custom extension folder: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#autoloading-processors This will make both the 1.16 and 1.22 versions of the mongo processor available for use in your 1.22 Apache NiFi. The Mongo Driver has been updated numerous times between 1.16 and 1.23: https://issues.apache.org/jira/browse/NIFI-9886 https://issues.apache.org/jira/browse/NIFI-10557 https://issues.apache.org/jira/browse/NIFI-11419 https://issues.apache.org/jira/browse/NIFI-11856 If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@SAMSAL  The managed Authorizer uses the file-access-policy-provider (generates the authorizations.xml if it does no already exist) and then a user-group-provider.  In your case that would make most sense to be the ldap-user-group-provider.  You may also want to use the Composite-configurable-user-group-provider (configure it with ldap-user-group-provider and file-user-group-provider).  Having both a file based provider and ldap provider allows sycning of ldap users and groups  form ldap automatically as well as the file provider allowing you to manually add non ldap user/client identities for authorization as well.    Non ldap client/user identities might be certifcate based clients like other NiFi nodes/instance, etc.. Within the file-access-policy-provider you define the initial admin identity. That user identity could be set to your ldap user account identity.  Then on first start up with managed provider, it generates the authorizations.xml file seeded with the policies necessary for that initial admin user identity to act as admin.  So you could skip the single-user-provider step. Matt ... View more

@edtech  And the ListenSyslog and putSyslog processors supports UDP: If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt   ... View more

@ChuckE  Plus 1 on @SAMSAL post.  I also had no problem with it working on my CFM 2.1.5.3 SP3 (Based off Apache NiFi 1.18 with many bug fixes from 1.19 - 1.20)      Thanks, Matt ... View more

@hegdemahendra  You also need to be careful with some rest-api request as they may generate reports that are held in NiFi heap until a rest-api call is made to remove them.  Doing similar actions directly from the UI handles the multiple calls needed automatically.  So calls like these can slowly eat away at the NiFi heap impacting performance until a restart.  Provenance queries would be an example of this. Unrelated note:  NiFi Variable registry has been deprecated in favor of NiFi Parameters now.  The Variable registry functionality has been officially removed starting with the latest NiFi 2.0.0-M1 release.   If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more

@Rohit1997jio  You could use the RetryFlowFile processor for this use case.  You will feed the "failure" relationship via a connection to the RetryFlowFile processor.  The RetryFlowfile processor will continue to route the FlowFile back to PublishKafka using the "retry" relationship until maximum number of retries configured has been exceeded.  After max retries has been reached the FlowFile would instead route to the "retries_exceeded" relationship which you can connect to a LogMessage processor.  The logMessage processor would then auto-terminate the "success" relationship. The challenge you have here is  your requirement to retry once per hour for 24 hours.  You could set the penalty duration in the PublishKafka to 1 hour.  This means that FlowFile routes to the "failure" relationship would get penalized for 60 mins.  The RetryFlowFile would not consume that FlowFile from input connection until penalty duration ended.  Then configure your number of retries in the RetryFlowFile processor to 24.  Be careful with setting queue size to 250 on the failure connection.  If you reach 250 queued on the failure relationship, it will trigger backpressure on the PublishKafka processor meaning the publishKafka processor would not get scheduled again until that backpressure is gone. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt ... View more