Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Nifi PutSNS - Size Exceeds 256KB

avatar
New Contributor

I'm using a PutSNS processor which puts JSON (from API call) into SNS, in some cases JSON is more than 256KB, and I get "Size exceeds AWS SNS Limit of 256KB". Is there a way to chunk this into smaller sets?

1 ACCEPTED SOLUTION

avatar
Super Guru

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect.


Check out this way to do SNS up to 2gb per payload:

 

https://github.com/awslabs/amazon-sns-java-extended-client-lib

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven

View solution in original post

1 REPLY 1

avatar
Super Guru

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect.


Check out this way to do SNS up to 2gb per payload:

 

https://github.com/awslabs/amazon-sns-java-extended-client-lib

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven