Support Questions

Find answers, ask questions, and share your expertise
Announcements
Check out our newest addition to the community, the Cloudera Data Analytics (CDA) group hub.

Zip S3 files together using NIFI

Explorer

Hi,

 

How can I zip s3 files together and drop them into another S3 folder? the s3 files have different formats and layouts, so merge content is not an option and also those S3 files are generated by upstream flow files so they are dynamically generated so need some kind of correlation id based zipping of s3 files together.

 

Thanks for community help.

2 REPLIES 2

Super Guru

@budati 

In mergeContent processor we can add correlation id property value then use MergeFormat as Zip to zip the files.

Then by using PutS3Object to store files into S3.

Explorer

I need to zip 3 files from S3, so will the flow be like  3  FetchS3Object  processors (  for 3 files )  → Merge Content → PutS3Object ? 

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.