Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

ListFile to list all the files sorted by datetime

avatar
New Contributor

How do I use ListFile to process the files in datetime order , I need to process the files in chronological order, oldest file first ?

1 ACCEPTED SOLUTION

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
3 REPLIES 3

avatar
Master Guru

@Raja M

List file processor with default configurations already does what you are expecting(lists/sends oldest file first) unless if you configured sucess queue with some kind of prioritizers.

ListFile is a stateful processor and creates a flowfile/s that represents the file/s based on the last modified time and default queue prioritizer is OldestFlowfileFirst prioritizer.

if you are using default queue configs then the processor will send first the oldest flowfile in the data flow will be processed first.

Available NiFi queue prioritizers are as follows:

  • FirstInFirstOutPrioritizer: Given two FlowFiles, the one that reached the connection first will be processed first.
  • NewestFlowFileFirstPrioritizer: Given two FlowFiles, the one that is newest in the dataflow will be processed first.
  • OldestFlowFileFirstPrioritizer: Given two FlowFiles, the one that is oldest in the dataflow will be processed first. 'This is the default scheme that is used if no prioritizers are selected.'
  • PriorityAttributePrioritizer: Given two FlowFiles that both have a "priority" attribute, the one that has the highest priority value will be processed first. Note that an UpdateAttribute processor should be used to add the "priority" attribute to the FlowFiles before they reach a connection that has this prioritizer set. Values for the "priority" attribute may be alphanumeric, where "a" is a higher priority than "z", and "1" is a higher priority than "9", for example.

To get to know about which files are transferred from the list file processor use Control Rate processor with

Rate Control Criteria

flowfile count

Maximum Rate

1

Now you can view in the success queue of Control Rate processor which flowfiles are released from the list file processor.

-

If the Answer helped to resolve your issue, Click on Accept button below to accept the answer, That would be great help to Community users to find solution quickly for these kind of issues.

avatar
Super Mentor
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
New Contributor

Yes.. I have to select "OldestFlowFileFirstPrioritizer" to make it work. Thank you @Matt Clarke and @Shu