Hi @MattWho We are pursuing cloudera to get a backport of this feature in the current release, as we are unable to upgrade hdf to a version having this feature. Meanwhile I am trying to implement the same using the ExecuteScript(penalize flow file), and eliminate duplicates. When trying to eliminate the duplicates, the detectduplicate does not work as expected when 2 different file names are introduced in the flowfile. Is this expected behaviour? How do we handle this scenario to have unique file list of multiple files over a period of time. The first generateflowfile sets filename as file1.txt and second generateflowfile sets filename as file2.txt. When i disable one of the generateflowfile, it is filtering the duplicates as expected. But when both the generateflowfile are running, we are getting duplicates. Thank you
... View more
