skip directories using nifi getftp processor

[#1]

Dears,

We need to skip the entire directory MATFIC, as this directory contains in average of 6K sub-directories (for having good response time of getFTP processor we need to skip this scanning operation).

Using the java regexpr tests ^((?!MATFIC).)*$ works, but in the getFTP processor it's not the case. We cannot use the listFTP because is based on the last modified date, which can cause we can skip some files that are sent to our FTP.

Hope this clarifies.

Would it be an other option to exclude this directory ? (Maybe an other regex?)

Is it a bug on getftp regex part ?

Thanks.

[#2 · Accepted solution]

@Bilel Boubakri,

I've tested your use case with getSFTP processor, it worked as expected with given regex (skipped MATFIC dir with all the files in it). I cannot test specifically GetFTP, as we don't have FTP servers running, but I'm sure it should give the same result. Tested with NiFi 1.1.0.2.1.1.0-2.

Some suggestions:

1. if you copy-pasted your regex - make sure you don't have new line character

2. make sure your regex is in Path Filter Regex and not in File Filter Regex property

3. If 1 and 2 don't help, try to dance with tambourine and call Cthulhu

Hope, that will help!