Skip to main content
MKS_AWS
MKS_AWS
Bronze ⭐️⭐️⭐️
September 23, 2020
Solved

Nifi PutSNS - Size Exceeds 256KB

I'm using a PutSNS processor which puts JSON (from API call) into SNS, in some cases JSON is more than 256KB, and I get "Size exceeds AWS SNS Limit of 256KB". Is there a way to chunk this into smaller sets?

stevenmatison
stevenmatisonAnswer
Platinum ⭐️⭐️
September 24, 2020

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect.


Check out this way to do SNS up to 2gb per payload:

 

https://github.com/awslabs/amazon-sns-java-extended-client-lib

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven

View in thread

1 reply

stevenmatison
stevenmatisonAnswer
Platinum ⭐️⭐️
September 24, 2020

@MKS_AWS There are a few ways to break up JSON within a flowfile (splitJson,QueryRecord).  However if its just one giant blob of JSON you may not find that very useful.    Perhaps you can share some sample json to that effect.


Check out this way to do SNS up to 2gb per payload:

 

https://github.com/awslabs/amazon-sns-java-extended-client-lib

 

 

If this answer resolves your issue or allows you to move forward, please choose to ACCEPT this solution and close this topic. If you have further dialogue on this topic please comment here or feel free to private message me. If you have new questions related to your Use Case please create separate topic and feel free to tag me in your post.  

 

Thanks,


Steven

Translate
    Terms & ConditionsAccessibility statement