Support Questions
Find answers, ask questions, and share your expertise

storing JSON records in S3

Expert Contributor

My nifi workflow is like this:

1. Generate Table Fetch

2. Extract Text

3. ExecuteSQL

4. SplitAVRO

5.AvroToJSON

6. Put S3

Now, in, put s3, object key by default in ${filename}.

It appears to me that records that were in the same file are getting overwritten. How do I handle this?

Ofcourse I need them all, preferably in the same file that were in same file before splitting.

1 REPLY 1

Re: storing JSON records in S3

Super Guru

@Simran Kaur,

if the flowfiles are overwriting then you need to use object key as

${UUID()}

instead ${filename}. This UUID is uniquely generated you will not have overwriting issues anymore.