Nifi PutSNS - Size Exceeds 256KB

MKS_AWS — Wed, 23 Sep 2020 21:40:13 GMT

I'm using a PutSNS processor which puts JSON (from API call) into SNS, in some cases JSON is more than 256KB, and I get "Size exceeds AWS SNS Limit of 256KB". Is there a way to chunk this into smaller sets?

Re: Nifi PutSNS - Size Exceeds 256KB

stevenmatison — Thu, 24 Sep 2020 13:34:49 GMT

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord). However if its just one giant blob of JSON you may not find that very useful. Perhaps you can share some sample json to that effect.

Check out this way to do SNS up to 2gb per payload:

https://github.com/awslabs/amazon-sns-java-extended-client-lib

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.

Thanks,

Steven

question Re: Nifi PutSNS - Size Exceeds 256KB in Support Questions

Nifi PutSNS - Size Exceeds 256KB

Re: Nifi PutSNS - Size Exceeds 256KB