Created 04-19-2018 08:50 PM
My Current flow : ListenHTTP-->EvaluateJsonPath-->ExecuteScript-->PutFile.
So far I am getting file path in Text file e.g. C:/user/usr_num/tmp (Location provided in PutFile processor), now I want to fetch all file from that location(e.g. C:/user/usr_num/tmp) and move it to another location(e.g. C:/user/usr_num/tmp/nifi_output). How would i do that?
Created on 04-19-2018 11:27 PM - edited 08-17-2019 07:32 PM
@steve Pat
Once your receive Text File then use Extract Text Processor to extract the file path from the text file.
Sample Flow:-
1. get the file from invokehttp...etc
2.Extract Text Processor //extract file path from text file
3.ExecuteStream Command //list out all the files in the path. feed outputstream relation to split text processor
4.SplitText //split the file each line into individual flowfile. feed splits relation to extract text processor
5.Extract Text //extract the filename from the flowfile
6.FetchFile //use absolute.path, filename to fetch and move the file to destination directory
Add new property with the matching regular expression that can extract the file path from the file.
Extract Text Processor Configs:-
Assuming that your file just having the file path so i added new property as
absolute.path
(.*)
We have extracted the file path and added as attribute name absolute.path to the flowfile.
Now the flowfile will have attribute called absolute.path with value as C:/user/usr_num/tmp.
ExecuteStreamCommand processor:-
This processor will be used to list out all the files that are in the C:/user/usr_num/tmp directory.
For windows configs:-
Command Arguments
/C;dir /b /a-d;${absolute.path}
Command Path
cmd
Ignore STDIN
false
Argument Delimiter
;
For Linux Configs:-
Now we are going to get all the files in the directory as a OutputStream flowfile.
Split Text Processor:-
Feed OutputStream relationship to Split text processor.
SplitText Configs:-
now we are splitting the outputstream flowfile which is having all the filenames in it to each filename in one flowfile.
ExtractText Processor:-
Feed the splits relation of SplitText processor to Extract Text processor and add new property as
filename
(.*)
In this extract text processor we have extracted filename and first extract text processor we have extracted absolute.path
We are having both absolute.path attribute and filename attribute to each flowfile.
Fetch File Processor:-
Use fetch file processor to fetch the files from your input directory and move the files to C:/user/usr_num/tmp/nifi_output directory
Configs:-
we are fetching the file and moving the file to nifi_output directory once the fetch is completed. Then auto terminate success relationship of Fetch File processor as we have already moved the file to our nifi_output directory.
(or)
Keep the Fetch File processor config
Completion Strategy as either of below
None //leaves the file as is
(or)
delete file //deletes the file from file system once fetch is completed
now the file is fetched from the directory then use PutFile processor and configure put file processor with your desired output C:/user/usr_num/tmp/nifi_output directory.
---
By following above steps you are able to move the all the files from one directory to other directory.
Let us know if you have any questions ..!!
Created 04-23-2018 07:53 AM
cc: @Steve Pat as @Shu accidentally mentioned me by mistake 🙂