Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

How use ListSFTP to find particular file when same name on different path

avatar
New Contributor

our files:

path/sas/a.text

path/sas/b.text

path/sas//bas/a.text

path/sas/bas/b.text

 

how use File Filter Regex and Path Filter Regex to find path/sas/a.text and path/sas/bas/b.text ?

there are my filter, I always got four files.

File Filter Regex:  a.text|b.text

Path Filter Regex: path/sas|path/sas/bas

1 ACCEPTED SOLUTION

avatar
Super Mentor

@Allen123 

 

Based on your description, the ListSftp processor is working exactly as designed and configured.

You have told it to list file from both directories and have also told it to search for both filenames in those directories. There is no way to configure the processor to ignore the extra unwanted a.text or b.text files.

You have two options:
Option 1:
- Create to listSFTP processor with each configured to list within a unique directory and "Search Recursively" set to false.  Then set a "File Filter Regex" so it only lists the desired file from each directory.
- The feed the success relationship from each of those two ListSftp processors to a single FetchSftp processor.

Option2:
The ListSftp processor is designed to only list files and not actually consume any content.  The content of the files listed is consumed later using the FetchSftp processor.  Between the ListSftp and the FetchSftp processors you could insert a RouteOnAttribute [1] processor that you could configure to only pass the two specific files you want on to FetchSftp while auto-terminating the others.

MattWho_0-1620676538485.png

MattWho_1-1620676636235.png

MattWho_2-1620676649152.png

 

You would route the matched relationship via a connection to the FetchSFTP and "auto-terminate" the unmatched relationship.

[1] http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache...


If you find that this has helped with your query, please take a moment to login and click accept on the solution.

Thanks,

Matt

View solution in original post

2 REPLIES 2

avatar
Super Mentor

@Allen123 

 

Based on your description, the ListSftp processor is working exactly as designed and configured.

You have told it to list file from both directories and have also told it to search for both filenames in those directories. There is no way to configure the processor to ignore the extra unwanted a.text or b.text files.

You have two options:
Option 1:
- Create to listSFTP processor with each configured to list within a unique directory and "Search Recursively" set to false.  Then set a "File Filter Regex" so it only lists the desired file from each directory.
- The feed the success relationship from each of those two ListSftp processors to a single FetchSftp processor.

Option2:
The ListSftp processor is designed to only list files and not actually consume any content.  The content of the files listed is consumed later using the FetchSftp processor.  Between the ListSftp and the FetchSftp processors you could insert a RouteOnAttribute [1] processor that you could configure to only pass the two specific files you want on to FetchSftp while auto-terminating the others.

MattWho_0-1620676538485.png

MattWho_1-1620676636235.png

MattWho_2-1620676649152.png

 

You would route the matched relationship via a connection to the FetchSFTP and "auto-terminate" the unmatched relationship.

[1] http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.13.2/org.apache...


If you find that this has helped with your query, please take a moment to login and click accept on the solution.

Thanks,

Matt

avatar
New Contributor

Thanks, it's help me a lot.