Member since
07-30-2019
3142
Posts
1566
Kudos Received
911
Solutions
My Accepted Solutions
Title | Views | Posted |
---|---|---|
56 | 01-21-2025 06:16 AM | |
114 | 01-16-2025 03:22 PM | |
206 | 01-09-2025 11:14 AM | |
1260 | 01-03-2025 05:59 AM | |
515 | 12-13-2024 10:58 AM |
01-25-2025
09:17 AM
Distinguish NiFi UI canvas by environments—Dev, Test, Prod—using color-coded labels, banners, or unique background images. Capture and edit these visual distinctions with Lightroom Pro. Its advanced editing tools allow you to fine-tune colors and overlays, creating clear, professional visuals for seamless environment identification and improved workflow clarity. @MattWho @Raj_B
... View more
01-25-2025
06:07 AM
1- yes i tried the last version 2- yes it does 3-yes i can and it fetch lists in nifi when i use the timestamp not the entities list 4- yes , it does 5- no change 6-yes 7-there is no exception 8- yes it does
... View more
01-24-2025
07:01 AM
@AbhiTryingAgain Welcome to the community and to Apache NiFi. Before building Dataflows it is important to understand the basics of a NiFi FlowFile. This will help you navigate the available processor components and the expectations on what they do at a high level. NiFi utilizes FlowFile so that it can remain data agnostic allowing NiFi to handle content of any type. Now performing actions against the content of a FlowFile would require processors that can understand the data format of the content. A NiFi FlowFile is what is transferred from one NiFi Processor on the canvas to the next. A FlowFile consists of two parts: FlowFile Content- This is the actual binary data/content. NiFi persist this content in content claims within the NiFi content_repository. FlowFile Attributes/Metadata - This is attributes and metadata about the content and FlowFile. At the most basic level, all FlowFiles will have timestamps, filename, etc attributes. Various NiFi processor will add even more attributes to a FlowFile. Understanding above, NiFi processors like "RouteOnAttribute" never look at the content of a FlowFile, it only looks at the FlowFile attributes of a FlowFile and route the FlowFile to the specified dynamic downstream relationship. So when you setup three routes, they all evaluated to 'true' and FlowFile was cloned and routed to all three downstream relationships. What you need is a NiFi processor that will evaluate the multiple json records in your FlowFile's content and output multiple FlowFiles based on a unique category value. For this, I think the PartitionRecord processor is whet you could use. This avoids splitting you Json record in to multiple records and then merging the various splits back into single FlowFiles based on category. You can then use the JsonTreeReader and JsonRecordSetWriters. Based on your example, configurations would look like this: Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-23-2025
11:32 PM
Thanks for the prompt response. We are working to get the stack-trace logs and would get back to you. Thanks, Amit.
... View more
01-23-2025
10:27 AM
@spartakus You will want to review the release notes for all release between 1.23. and 1.28.1 https://cwiki.apache.org/confluence/display/NIFI/Release+Notes/#ReleaseNotes-Version1.23.0 You'll notice a number of components deprecated. Deprecated components are not removed, but that are marked as deprecated and will log if used that they are deprecated. Deprecated components will eventually be removed form Apache NiFi (most removed in NiFi 2.0). You'll also make note of any items in the release notes that state: Moved <XYZ> to optional build profiles This means that these components were removed from Apache NiFi. These components can be added manually later by downloading the nars from the central maven repository and adding them to the NiFi lib directory if you need them still. https://mvnrepository.com/artifact/org.apache.nifi Make a back up of your flow.json.gz, nifi.properties, and optionally flow.xml.gz files before you upgrade. Otherwise, I see no issues with upgrading from 1.22 -- > 1.28 directly. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-23-2025
02:07 AM
@itninja, Did the response help resolve your query? If it did, kindly mark the relevant reply as the solution, as it will aid others in locating the answer more easily in the future. However, if you still have concerns, could you please provide the information that @MattWho has requested?
... View more
01-22-2025
08:42 AM
@anon12345 Yes, you need to unset the keystore and truststore properties because NiFi will still attempt to load them even if HTTPS properties are unset. This is by design because NiFi can act as a client and as a client it may attempt to connect to secured endpoints where NiFi would use this keystore and trustore. So ay property configured in nifi.properties will be loaded, so they need to be unset or have valid values set. For example: NiFi's Site-To-Site capability. A unsecure NiFi configured with a valid keystore and truststore can have a Remote Process Group configured to connect to a Remote port on another secured NiFi. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-21-2025
06:16 AM
@jirungaray Cloudera Flow Management (Based on Apache NiFi) provides multiple methods for managing user authorization. This includes NiFi internally via the File-Access-Policy-Provider and externally via Apache Ranger. There is no built in mechanism for auto setting up authorization policies for users or groups with the exception of the Initial Admin and Initial NiFi Node authorizations. Many of the Authorization policies are directly related to the components added to the canvas. Those components are assigned unique IDs making it impossible to create policies before the components exist. File-Access-Policy-Porvider: This provider utilizes a file on disk (authorizations.xml) to persists authorization policies. This file is loaded when NiFi starts. This means it is possible to manually generate this file and have NiFi load it on startup. Also as you mentioned, you could script out the authorization creating through NiFi Rest-API calls. Ranger provider: This moves authorization responsibility over to Apache Ranger. Policies setup within Ranger are download by the NiFi nodes where they are locally enforced. No matter which authorizer you choose to use, authorizations are easiest to manage via groups. Typical users setup ldap groups for various NiFi roles (admins, team 1, team2, etc..) and makes specific users members of these groups. This simplifies authorization since you can authorizer these groups instead of the individual users. Simply adding or removing a user as member of one of these authorized groups gives or removes authorized access to the NiFi resource identifier (NiFi policy). The ldap-user-group-provider can be added to the NiFi authorizers.xml to auto manage syncing of user and group identities from your AD/LDAP further simplifying management over the file-user-group-provider method which requires the manual adding of user and group identifiers to the NiFi. Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped. Thank you, Matt
... View more
01-17-2025
07:37 AM
@MattWho Thank you for explanation. Now I understand MergeRecord determines to which file each flowfile to be merged by schema information. I'll consider increasing "Minimum number of records" as you recommended. Thanks,
... View more