Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Ftp sub-directories based on specific file on each sub-directory

Solved Go to solution

Ftp sub-directories based on specific file on each sub-directory

New Contributor

I have a remote directory as an input (containing sub-directories) and i'd like to iterate for each subdirectory and check if a specific file exist, once the last exist i will ftp this sub-directory to local directory using Nifi.

could you please help me on this scenario.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Ftp sub-directories based on specific file on each sub-directory

@Bilel Boubakri

I am assuming you know how many subdirectories there are in this solution.

ListFTP - scanning the main directory with all the subdirectories

EexcutScript - running a groovy script waiting for all of the flag.ok files to be in the subdirectories

FetchFTP - configured to pull the wanted files

Here is a snapshot of the simple flow:

19616-screen-shot-2017-07-27-at-50453-pm.png

Here is the sample groovy script for the ExecuteScript processor, configured for this example to wait for 5 files:

def flowFiles = session.get(5)
if(!flowFiles || flowFiles.size() < 5) {
  session.rollback()
} else {
  session.transfer(flowFiles, REL_SUCCESS)
}

waiting-for-5-files-before-ftping-the-files.xml Here is a template of the sample flow

3 REPLIES 3

Re: Ftp sub-directories based on specific file on each sub-directory

@Bilel Boubakri

Are you saying you want to wait until each sub-directory has a specific file and then start ftping the files to a local directory?

Re: Ftp sub-directories based on specific file on each sub-directory

New Contributor

@Wynner @Matt Clarke @Matt Burgess @Pierre Villard

I want to check each sub-directory once a specific file come like "flag.ok", i start ftping the files

of this sub-directory to a local directory() (always i need to loop all sub-directories

and doing the same job).

Thanks a lot for your support.

Re: Ftp sub-directories based on specific file on each sub-directory

@Bilel Boubakri

I am assuming you know how many subdirectories there are in this solution.

ListFTP - scanning the main directory with all the subdirectories

EexcutScript - running a groovy script waiting for all of the flag.ok files to be in the subdirectories

FetchFTP - configured to pull the wanted files

Here is a snapshot of the simple flow:

19616-screen-shot-2017-07-27-at-50453-pm.png

Here is the sample groovy script for the ExecuteScript processor, configured for this example to wait for 5 files:

def flowFiles = session.get(5)
if(!flowFiles || flowFiles.size() < 5) {
  session.rollback()
} else {
  session.transfer(flowFiles, REL_SUCCESS)
}

waiting-for-5-files-before-ftping-the-files.xml Here is a template of the sample flow