Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Zip S3 files together using NIFI

Zip S3 files together using NIFI

New Contributor

Hi,

 

How can I zip s3 files together and drop them into another S3 folder? the s3 files have different formats and layouts, so merge content is not an option and also those S3 files are generated by upstream flow files so they are dynamically generated so need some kind of correlation id based zipping of s3 files together.

 

Thanks for community help.

2 REPLIES 2
Highlighted

Re: Zip S3 files together using NIFI

Super Guru

@budati 

In mergeContent processor we can add correlation id property value then use MergeFormat as Zip to zip the files.

Then by using PutS3Object to store files into S3.

Re: Zip S3 files together using NIFI

New Contributor

I need to zip 3 files from S3, so will the flow be like  3  FetchS3Object  processors (  for 3 files )  → Merge Content → PutS3Object ?