Support Questions
Find answers, ask questions, and share your expertise

Transfer files based on date present in filename

Highlighted

Transfer files based on date present in filename

Contributor

Used case : Transfer files from inbound directory based on date condition in filename to destination folder with date .

 

Ex : Input directory :/user/mft/inbound/ 

       Filename : etfbfo_20200331.csv , gfsdajaf_20200401.csv, gdfcbakfg_20200402.csv.

 

Destination : /users/aws/outbound/20200331/ tfbfo_20200331.csv

                     /users/aws/outbound/20200401/ gfsdajaf_20200401.csv

                     /users/aws/outbound/20200402/gdfcbakfg_20200402.csv.

 @stevenmatison 

2 REPLIES 2

Re: Transfer files based on date present in filename

Expert Contributor

Hi @Gubbi 

You can try the below script.

 

source_dir="/user/mft/inbound/"
dest_dir="/users/aws/outbound/"

for file in "$source_dir"*; do dest=$(echo $file | grep -o -P '(?<=_).*(?=.csv)'); mkdir -p "$dest_dir$dest"; mv $file "$dest_dir$dest"; done

 

Hope this helps. Please accept the answer and vote up if it did.

Highlighted

Re: Transfer files based on date present in filename

@Gubbi   I think these topics are related?   The idea you need to accomplish here is getting all the files.  Then once all the files are in nifi,  check the details of the filename to get the date from there instead of using now() and comparing the filename to some version of "now" or "now minus X days".  

 

Once you have the files you just work off the filename to get the date:

date = ${filename:replace('.csv',''):substringAfter('_')}

  ^^ I did not test this, so you may have to adjust the nifi expression language to get the exact date match

 

Then use the ${date} in the path when you store the file.   This method removes the requirement to have different routes based on the date matching, and everything goes the same route with the dynamic path based on ${date}.

 

 

Hope this helps.