Created 10-24-2017 09:11 AM
Dears,
We need to skip the entire directory MATFIC, as this directory contains in average of 6K sub-directories (for having good response time of getFTP processor we need to skip this scanning operation).
Using the java regexpr tests ^((?!MATFIC).)*$ works, but in the getFTP processor it's not the case. We cannot use the listFTP because is based on the last modified date, which can cause we can skip some files that are sent to our FTP.
Hope this clarifies.
Would it be an other option to exclude this directory ? (Maybe an other regex?)
Is it a bug on getftp regex part ?
Thanks.
Created 10-25-2017 06:09 PM
I've tested your use case with getSFTP processor, it worked as expected with given regex (skipped MATFIC dir with all the files in it). I cannot test specifically GetFTP, as we don't have FTP servers running, but I'm sure it should give the same result. Tested with NiFi 1.1.0.2.1.1.0-2.
Some suggestions:
1. if you copy-pasted your regex - make sure you don't have new line character
2. make sure your regex is in Path Filter Regex and not in File Filter Regex property
3. If 1 and 2 don't help, try to dance with tambourine and call Cthulhu
Hope, that will help!
Created 10-25-2017 06:09 PM
I've tested your use case with getSFTP processor, it worked as expected with given regex (skipped MATFIC dir with all the files in it). I cannot test specifically GetFTP, as we don't have FTP servers running, but I'm sure it should give the same result. Tested with NiFi 1.1.0.2.1.1.0-2.
Some suggestions:
1. if you copy-pasted your regex - make sure you don't have new line character
2. make sure your regex is in Path Filter Regex and not in File Filter Regex property
3. If 1 and 2 don't help, try to dance with tambourine and call Cthulhu
Hope, that will help!