Member since
07-30-2019
3471
Posts
1642
Kudos Received
1020
Solutions
My Accepted Solutions
| Title | Views | Posted |
|---|---|---|
| 138 | 06-03-2026 06:06 PM | |
| 456 | 05-06-2026 09:16 AM | |
| 818 | 05-04-2026 05:20 AM | |
| 491 | 05-01-2026 10:15 AM | |
| 619 | 03-23-2026 05:44 AM |
06-14-2021
06:20 AM
@Rupesh_Raghani NiFi was not designed to provide a completely blank canvas to each user. There are important design reason for this. NiFi runs within a single JVM. All dataflows created on the canvas run as the NiFi service user and not as the user who is logged in. This means that all user's dataflows share and compete for the same system resources. Another user's poorly designed dataflow(s) can have an impact on the operation of another user's dataflow(s). So it is important for one users to be able to identify where backlogs may be forming even if that is occurring in another user's dataflow(s). With a secured NiFi, authorization policy control what a successfully authenticated user can see and do on the NiFi canvas. While components added to the canvas will always be visible to all users, what is displayed on the component is limited only stats for unauthorized users (no component names, component types, component configurations, etc). So an unauthorized user would be unable to see how that unauthorized component is being used and for what. The unauthorized user would also not have access to modify the component, access FlowFiles that traversed those components (unless that data passed through an authorized component somewhere else in the dataflow(s)), etc. Besides resource usage, another reason users need to see these place holders for all components is so that users do not build dataflows atop one another. It is common for multiple teams to be authorized to work within the same NiFi. It is also common to have some users who are members of more than one team. For those users, it would be very difficult to use the UI if each teams flows were built on top of one another. Most common setup involves an admin user creating a single Process Group (PG) on the root canvas level (top level - what you see when you first log in to a new NiFi). Then each team is authorized only to their assigned PG. So team1 user logs in and there PG is fully rendered and non authorized PGs are present by non configurable and no displayed details. team1 is unable to add components to canvas at this level and must enter their authorized PG before they can start building dataflows. When you enter sub-PG, you have a blank canvas to work with. Hope this helps with your query. Matt
... View more
06-13-2021
10:58 PM
@AnkushKoul , did @MattWho's response resolve your issue? If so, can you please mark it as the solution? It will make it easier for others to find the answer in the future.
... View more
06-08-2021
06:41 AM
@ang_coder The RouteOnAttribute processor establishes a NEW relationship for each dynamic property you add. If you intent is that a single FlowFile must satisfy all conditions to route on, then you should have just one NiFi Expression Language (NEL) statement that covers all conditions resulting a true or false boolean. If you share your two statements, I'd be happy to help you construct a single NEL statement. butt would be structured something like something like: All 4 lines would need to result in true before FlowFile would be related to this dynamic property's relationship. If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more
06-08-2021
06:07 AM
@techNerd The PutSFTP processor contains the following configuration property: Do you have that set to false on the particular putSFTP processor throwing the exception? Thanks, Matt
... View more
06-07-2021
08:34 AM
@Acbx It looks like your CSV uses commas as the field delimiter. So the solution i provided parses the entire file line by line and changes all "." to ",". So, I am guessing that you have other places within your CSV that also had ".", thus creating the additional 5 field columns. Are trying to create a new column for cents? Is that why you are changing 109.29 tp 109,29? If you are not looking for a new column, how will downstream system parse this edited CSV now that you added a new comma in there? You could write a complex Java regular expression in the Search Value to match only specifically on column number X (Money Column) and then use Replacement Strategy "Regex Replace" to edit it. Let's assume the "Money" Column was column number 5. And then wrap money once converted from 109.29 to 109,29 in quotes so it is not treated as two columns later on.... Search Value: ^(.*?),(.*?),(.*?),(.*?),(.*?),(.*?)$ Replacement Value: $1,$2,$3,$4,"${'$5':replace(".",",")}",$6 So above would manipulate column 5 only and change 109.29 in to "109,29". Hope this helps you, Matt
... View more
06-07-2021
04:21 AM
Thanks for both replies, I managed to get it working last week the same way as you have shown Matt. Cheers
... View more
06-04-2021
01:44 PM
@khaldoune Some components that maintain state do so because they were developed with intent of being used in NiFi cluster setup to support non cluster friendly protocols. Example (getting data from SFTP server): In Standalone NiFi you would use the GetSFTP processor (does not record state). In Cluster NiFi you would use the ListSFTP (records state) and FetchSFTP processor to do the same task. The ListSFTP processor would be configured to execute on "primary node" only. That way you do not have every node in your cluster trying to list the same files on yoru target SFTP processor. Then the success from listSFTP which simply has FlowFiles with no content and only metadata/attributes is connected to a FetchSFTP processor. That connection between those two processors would be configured to load balance those FlowFiles to all nodes. Now the heavy work of ingesting the actual content for each of those listed FlowFiles is spread across all nodes in the cluster. Even if you use above processors in a standalone, they will still record state. Cluster state is generally stored to help when a primary node change occurs. That way the newly elected primary node that now starts executing the primary node only configured processors, will have those processor fetch that last known state from ZK so that it does not list the same files already listed by previous primary node. Just some more context for you on how state is used primarily by components and why. Matt
... View more
06-03-2021
06:32 AM
Hi @MattWho , Thanks for the response. I'll happily create a feature request, though I wasn't sure if I was missing something obvious that would meet my objective/requirements. Thank you for clarifying, I'll go take a look at the links you provided (admittedly I had missed the mailing lists, oops). Kind regards Mark
... View more
06-03-2021
05:33 AM
@aie Has your issue been resolved? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future.
... View more
06-02-2021
07:52 AM
@midee I am not clearly following your use case. FlowFiles consist of two parts, FlowFile attributes/metadata and FlowFile content. You give example with "customefields_12345" and "customefields_12346". Does this mean one FlowFile may have multiple "customefields_<some string>" attributes assigned to it? How do you want to route FlowFiles where only some of those customfield attributes are null while others are not? There are multiple ways to handle this using NiFi Expression Language (NEL) [1] and the routeOnAttribute [2] processor. ${anyMatchingAttribute("customfield.*"):isEmpty()} Above would return "true" if ANY of the NiFi FlowFile attributes starting with "customefield" is empty. note: The isEmpty function returns true if the Subject is null, does not contain any characters or contains only white-space (new line, carriage return, space, tab), false otherwise. There is another NEL subjectless function that would return "true" only if ALL FlowFileAttributes matching the Java regular expression were empty: ${allMatchingAttributes("customfield.*"):isEmpty()} With the RouteOnAttribute processor you create/add dynamic properties and each of those becomes a new routable relationship on the processor. if the NEL statement configured for that dynamic property returns true that FlowFile routes to that relationship. Any FlowFile that does not return true for dynamic properties will get routed to the pre-existing relationship named "unmatched". [1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache.nifi.processors.standard.RouteOnAttribute/index.html If you found this addressed your query, please take a moment to login and click "Accept" on this solution. Thank you, Matt
... View more