I have a setup where there is 1 S3 bucket for each day and 4k files per bucket. The requirement I have is
1) Compress all the files
2) archive the files
3) push the files to different bucket
4) delete the source files after the success of the above three operations.
Please suggest a nifi flow for this requirement.
What I have tried:
I tried listing the files using ListS3, Fetching the files using FetchS3, Compress content and tar them using merge content processor. After merge I could not get the filenames to delete from source bucket.