Support Questions
Find answers, ask questions, and share your expertise

Nifi PutSNS - Size Exceeds 256KB

New Contributor

I'm using a PutSNS processor which puts JSON (from API call) into SNS, in some cases JSON is more than 256KB, and I get "Size exceeds AWS SNS Limit of 256KB". Is there a way to chunk this into smaller sets?

1 ACCEPTED SOLUTION

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect.


Check out this way to do SNS up to 2gb per payload:

 

https://github.com/awslabs/amazon-sns-java-extended-client-lib

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven

View solution in original post

1 REPLY 1

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect.


Check out this way to do SNS up to 2gb per payload:

 

https://github.com/awslabs/amazon-sns-java-extended-client-lib

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven

Take a Tour of the Community
Don't have an account?
Your experience may be limited. Sign in to explore more.