Member since
07-30-2019
3471
Posts
1642
Kudos Received
1020
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 143 | 06-03-2026 06:06 PM | |
| 458 | 05-06-2026 09:16 AM | |
| 823 | 05-04-2026 05:20 AM | |
| 493 | 05-01-2026 10:15 AM | |
| 620 | 03-23-2026 05:44 AM |
10-19-2021
01:18 PM
@AA24 NiFi was designed as an always on type of dataflow design. As such the NiFi processor components support "Timer Driven" and "Cron Driven" Scheduling Strategy types. That being said, the ability to tell a processor to "Run Once" exists within NiFi. You could manually do from within the UI by right clicking on the NiFi processor component and selecting "run once" from the pop-up context menu. The next thing to keep in mind is that anything that you can do via the UI, you can also do via a curl command. So it is possible to build a dataflow that could trigger the "run once" api call against the processor you want to fetch from the appropriate DB. You can not execute "run once" against a PG nor would I recommend doing so. You want to only trigger the file responsible for ingesting your data and leave all the other processor running all the time so they process whatever data they have queued at anytime. First you to create your trigger flow, so you could have a getFile to consume the trigger file and use maybe a RouteOnContent processor to send the FlowFile to either an InvokeHTTP configured to invoke run-once on your Oracle configured processor or an invokeHTTP configured to invoke run-once on your MySQL configured processor. Using your browser's developer tools is an easy way to capture the rest-api calls that are made when you manually perform them the action via the UI. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
10-12-2021
11:09 AM
Thanks @MattWho I took a more thorough look at the authorizers.xml file and I had a bad path to the users.xml file in the file-provider. There may still be some issues with the way this is being deployed, but I'm able to interact with the canvas now, so it's progress.
... View more
10-06-2021
12:53 PM
Hi Matt, Thank you for your suggestion I've recovered it by using flow.xml.gz file.
... View more
10-06-2021
09:29 AM
Thank you very much for your reply! I followed your advice and managed to get the processor working! Thanks again!
... View more
10-05-2021
11:33 PM
@TRSS_Cloudera Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
10-05-2021
01:07 PM
Hello @edoS Welcome to the community! NiFi's provides so many option for user authentication and authorization, setting up exactly what you need can be overwhelming at times. This is certainly something the Cloudera support could walk you through if you have a support contract with us that covers the NiFi service. At a high level, here is what you need to understand about the authentication and authorization process in NiFi. Authentication happens first and must be successful before any authorization is verified. NiFi supports numerous ways to authenticate users/clients (TLS, Kerberos, LDAP, openID, etc...). No matter which method is used, the end result of any authentication is a user string that identifies the successfully authenticated user/client. That user string is then evaluated against the identity mappings [1] you may have configured in the nifi.properties file. These identity mappings are used to normalize the user strings. for example: Trim the CN from the full DN in a user/client certificate Trim the user name from a kerberos principal convert the user string to all uppercase or lowercase The resulting user/client string is then passed to the authorizer to verify that user/client is authorized for the NiFi Resource Identifier being requested. NiFi authorizers.xml is where this configuration is setup. This file is easiest to read from the bottom up. At the bottom of the authorizers.xml you will find your authorizer which you have setup as the "Ranger-Provider". It is important to understand how this authorizer works. NiFi runs a background thread that checks in with Ranger to see if there is a new policy definition for the NiFi service. If so, the new definition is downloaded by NiFi. What Ranger provides to NiFi in this downloaded policy definition are all the polices setup in Ranger. For each there will be the "NiFi Resource Identifier(s)" along with the user strings and group strings that have been assigned "Read" and/or "Write" permissions. Now remember up to this point all NiFi knows about the authenticated user is the user string. NiFi has no idea yet what groups that user string may belong to. Within the Ranger-Provider, you will find a property name with "User Group Provider". The value set here tell the authorizer where to check to see if the user string passed from authentication has any known user to group associations. Search your authorizers.xml for configured User Group Provider [2]. There are numerous options that can be configured for determining user to group associations. Some of the available providers allow you to configured multiple providers. While the authorizer "ranger-provider" can only point at 1, it may point at a "composite-configurable-user-group-provider" [3] for example that can be setup to reference multiple user-group-providers. The key here is making sure you have added 1 or more user group providers that will return all the user to group associations you need. Based on the log output you shared from the nifi-user.log. We know that none of the user group providers you may have setup returned any group strings associated to your user string (identity[18330301],groups[] ). This is why "groups [ ]" is empty. The "file-user-group-provider" [4] allows you to create user string to group string associations manually via the NiFi UI directly. The commonly used "ldap-user-group-provider" [5] determines user and group associations via user and/or group syncs with ldap/AD. Now that NiFi knows what groups the authenticated user string is associated with, the user and the groups can be checked against the downloaded policies to see if the user is authorized for the action being performed or the end-point trying to be accessed. [1] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#identity-mapping-properties [2] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#authorizers-setup [3] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#composite-implementations [4] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#fileusergroupprovider [5] https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldapusergroupprovider If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more
10-05-2021
12:49 AM
Thanks @MattWho ! It works. The feature "ReportLineageToAtlas" is now enabled in NiFi. Thanks a lot.
... View more
10-03-2021
10:32 PM
Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
09-30-2021
02:03 PM
Thanks @MattWho for the quick response. I can see the active thread in the top right corner of the processor which is hang forever. The max timer driven threads are not consumed, i will try to get the thread dump and see. I think i should contact provider of the processor (marklogic) as you suggested. Thank you.
... View more
09-30-2021
06:10 AM
2 Kudos
@DSan You run in to some unique challenges with your specific use case since the properties you want to enable NEL expect Java regular expressions. The reserved characters that would tell NiFi that a NEL statement is being used are special characters in Java Regular expressions. You may want to raise your change/improvement request as a Jira in the Apache NiFi Jira project: https://issues.apache.org/jira/browse/NIFI There may be others in the community that like this idea and have suggestions for working around the challenge I shared above. If you found this response assisted with your query, please take a moment to login and click on "Accept as Solution" below this post. Thank you, Matt
... View more