https://community.cloudera.com/t5/Support-Questions/Filter-duplicate-files-in-NifI/m-p/305977#M222607 <P>Hello,</P><P>&nbsp;</P><P>You can use DetectDuplicate processor and only route non-duplicate to HDFS.</P><P>&nbsp;</P><P>But using the GETSFTP processor should work fine as you configured it to run on primary node only. What errors were you facing back then ?</P> Fri, 13 Nov 2020 16:49:21 GMT Kezia 2020-11-13T16:49:21Z