- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
EnforceOrder Inactive Timeout not working
- Labels:
-
Apache NiFi
Created 10-03-2024 06:15 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello
I am using the EnforceOrder in order to sort the flow files. This procesosr is maintaning state and i want to clear in it some regular interval for e.g. 4 min. I am runnig NIFI over a cluster. I am clearing the state through Inactive Timeout propery by setting it 4 min. I can see some time state is clear and most of the time not. Please help in a resolution for how clear state consistently.
Created 10-03-2024 08:51 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@varungupta
The EnforceOrder processor maintain state locally specific to each node in the cluster. When you right click on the processor an view state, you are seeing the reported retained local state form each node. The purge of state will only happen on a specific node where the criteria necessary for purge has been met.
Might be helpful to share your processor configuration here including properties and scheduling.
How are the Source FlowFiles being ingested and fed to this processor?
You can look at lineage on the EnforceOrder processor to see FlowFile history.
Thanks,
Matt
Created on 10-03-2024 09:14 AM - edited 10-03-2024 11:46 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi @MattWho
Here is the configuration attached. I am setting Inactive Timeout as 2 min. Though after 10 min still the view state is not clear. Please refer to both screenshots. Flow is as below
ListFile-->UpdateAttribute for Order attribute-->EnforceOrder--> Further processing through other processor
Created 10-04-2024 09:50 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
@varungupta
Tell me a bit more about your ListFile processor configuration.
- Is the input directory a shared mount across all your nodes or does each node have a unique set of files in the input directory?
- Is ListFile configured for execution of "All nodes" or "Primary node"?
- How many files are being listed when ListFile is executing (Is it less then 10,000)?
- How often is ListFile scheduled to run?
- Is List File traversing sub-directories for files?
I assume you are extracting the sequenceNo from the filename.
As far as Enforce Order goes:
- How are you handling the various relationships (routing via connection to where)?
- Are you seeing FlowFiles routed to "overlook" relationship?
Matt
