Transfer files based on date present in filename

[#1]

Used case : Transfer files from inbound directory based on date condition in filename to destination folder with date .

Ex : Input directory :/user/mft/inbound/ 

       Filename : etfbfo_20200331.csv , gfsdajaf_20200401.csv, gdfcbakfg_20200402.csv.

Destination : /users/aws/outbound/20200331/ tfbfo_20200331.csv

                     /users/aws/outbound/20200401/ gfsdajaf_20200401.csv

                     /users/aws/outbound/20200402/gdfcbakfg_20200402.csv.

 @stevenmatison 

[#2]

Hi @Gubbi 

You can try the below script.

source_dir="/user/mft/inbound/"
dest_dir="/users/aws/outbound/"

for file in "$source_dir"*; do dest=$(echo $file | grep -o -P '(?<=_).*(?=.csv)'); mkdir -p "$dest_dir$dest"; mv $file "$dest_dir$dest"; done

Hope this helps. Please accept the answer and vote up if it did.

[#3]

@Gubbi   I think these topics are related?   The idea you need to accomplish here is getting all the files.  Then once all the files are in nifi,  check the details of the filename to get the date from there instead of using now() and comparing the filename to some version of "now" or "now minus X days".  

Once you have the files you just work off the filename to get the date:

date = ${filename:replace('.csv',''):substringAfter('_')}

  ^^ I did not test this, so you may have to adjust the nifi expression language to get the exact date match

Then use the ${date} in the path when you store the file.   This method removes the requirement to have different routes based on the date matching, and everything goes the same route with the dynamic path based on ${date}.

Hope this helps.