Member since
07-30-2019
3391
Posts
1618
Kudos Received
1000
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 251 | 11-05-2025 11:01 AM | |
| 157 | 11-05-2025 08:01 AM | |
| 488 | 10-20-2025 06:29 AM | |
| 628 | 10-10-2025 08:03 AM | |
| 402 | 10-08-2025 10:52 AM |
03-12-2024
12:47 AM
Thank you Matt for all your help , really appreciate this I will try this and let you know
... View more
03-11-2024
08:45 PM
OK, thanks for the answer. It seems that the current solution can only be upgraded to the highest version of nifi that is compatible with Java8.
... View more
03-08-2024
02:20 AM
1 Kudo
Thank you so much @MattWho !
... View more
03-07-2024
12:15 PM
1 Kudo
@Chetan_mn Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.
... View more
03-05-2024
10:53 AM
Hello @MattWho , Thank you for the reply. Initially, I thought it would be best to include all the details in the ticket to avoid any confusion. Here's what I discovered: When the API was first executed, it fetched 100 records. Let's consider that all issues in JIRA contain different types of information, such as packages, bugs, epics, stories, and modules and they will have different columns . When fetching a small number of rows, the data appeared consistent, but when fetching data for 4-5 months, the data structure changed. For example, a bug might have 3 columns, a story might have 8, and epic might have 3. Consequently, all records had missing columns filled with null values. Here's an example: Bug 1: Columns A, B, C
Story 2: Columns A, B, C, D, E, F, G
Epic 3: Columns A, B As a result, the final records looked like this: {
A: Value,
B: Value,
C: Value,
D.:null,
E: null,
F: null,
G: null
},
{
A: Value,
B: Value,
C: Value,
D.:Value,
E: Value,
F: Value,
G: Value
},
{
A: Value,
B: Value,
C: null,
D.:null,
E: null,
F: null,
G: null
} To address this issue, I used the SplitJSON processor to split the records and process them individually . This resolved the issue. However, after implementing this solution, I encountered another issue where the choice list was not inserting records. I managed to handle this issue as well, and now everything is working fine.
... View more
03-05-2024
08:46 AM
Correct. A FlowFile might over its dataflow lifetime point at different content claims for its content. That all depends on the processors used in the dataflow. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
03-04-2024
07:30 AM
@MvZ The "file-login-provider" login identity-provider has never existed in any out-of-the-box release of Apache NiFi. If you have created or downloaded some custom implementation of this provider. You would need to consult with that author in getting it to work. Where did you obtain this provider from and what process did you follow to add it to your NiFi installation? The exception you have shared simply tells you that during startup NiFi is loading the nifi.properties file and the property "nifi.security.user.login.identity.provider" is configured with "file-login-provider"; however, when NiFi parsed the login-identity-providers.xml configuration file, no provider with: <identifier>file-login-provider</identifier> was found in that configuration file. I can't provide any guidance on this provider as I was unable to find anything online about what I am expecting is a custom add-on provider. The out-of-the-box available authentication providers are found in the NiFi documentation here: Apache NiFi 1.2x versions: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#user_authentication Apache NiFi 2.x versions: https://nifi.apache.org/documentation/nifi-2.0.0-M1/html/administration-guide.html#user_authentication NiFi Authentication and Authorization are two different configurations and independent configurations. Once you have chosen how you want to handle user authentication, you then move on to setting up user authorization: https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#multi-tenant-authorization. For file based authorization, NiFi offers two providers: 1. Older deprecated FileAuthorizer 2. The current StandardManagedAuthorizer These providers are configured in the NiFi authorizers.xml file. No direct useer policies get defined in the authorizers.xml file. The FileAuthorizer or the FileAccessPolicyProvider referenced by the StandardManagedAuthorizer will generate the initial authorizations.xml file with the initial admin user configured in the provider chosen. You would not typically manually generate or manipulate this file. Instead you would acces your NiFi's UI using that initial admin and define additional user authorizations directly via the NiFi UI. Here is an example of what you would have in your authorizers.xml if using the StandardManagedAuthorizer: <authorizers>
<userGroupProvider>
<identifier>file-user-group-provider</identifier>
<class>org.apache.nifi.authorization.FileUserGroupProvider</class>
<property name="Users File">./conf/users.xml</property>
<property name="Legacy Authorized Users File"></property>
<property name="Initial User Identity 1">ronald</property>
</userGroupProvider>
<accessPolicyProvider>
<identifier>file-access-policy-provider</identifier>
<class>org.apache.nifi.authorization.FileAccessPolicyProvider</class>
<property name="User Group Provider">file-user-group-provider</property>
<property name="Authorizations File">./conf/authorizations.xml</property>
<property name="Initial Admin Identity">ronald</property>
<property name="Legacy Authorized Users File"></property>
<property name="Node Identity 1"></property>
</accessPolicyProvider>
<authorizer>
<identifier>managed-authorizer</identifier>
<class>org.apache.nifi.authorization.StandardManagedAuthorizer</class>
<property name="Access Policy Provider">file-access-policy-provider</property>
</authorizer>
</authorizers> If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
02-26-2024
08:27 AM
3 Kudos
@krishna123 @jameswookyz @rafy NiFi processor are configured with a Run Schedule, by default processors are configured with a Run Schedule of 0 secs. This tells NiFi core to schedule this processor to execute as often as possible. The Scheduling part of the processor handles checking if any of the inbound connections to the processor with queued data or last execution resulted in data. If there is no inbound queued FlowFiles, the NiFi controller will yield the processor scheduling. This yielding is designed to prevent the processor from just constantly trying to schedule when there is no work to do. If there is work to do, the processor will get scheduled to execute. The scheduling typically consumes microseconds of CPU time. And the built-in yielding prevents excessive cpu usage when no work exists to execute upon. Adjusting the run schedule does not change behavior of yielding, but when flow is constant for periods of time, changing the run schedule alters the throughput performance. Hope this clarifies things. If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
02-26-2024
08:14 AM
@ShyamKumar Your dataflow design is still unclear here. - 'We have different client which will call the same generic PG" --> How is this being done? When you say "clients", are you referring to external to NiFi clients? How are these client request being sent to/received by NiFi's PG? We would need to understand your dataflow better before being able to provide better feedback. A detailed use case would be very helpful here. Thanks, Matt
... View more
02-26-2024
07:44 AM
@mechtech007 Providing details such as the exact version of CFM NiFi you are using is very important in getting help as NiFi evolves over time, those who help want to know as much as possible in order to avoid providing irrelevant feedback that may not pertain to the version you are using. You could rename the flow.xml.gz and flow.json.gz (present only if new enough version of CFM) and restart the node. The node will inherit the flow from the cluster nodes when it tries to join the cluster. Depending on how old your CFM version is, you could be hitting one of these bugs: https://issues.apache.org/jira/browse/NIFI-7920 https://issues.apache.org/jira/browse/NIFI-8204 If you found any of the suggestions/solutions provided helped you with your issue, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more