Support Questions

Find answers, ask questions, and share your expertise
Announcements
We’ve updated our product names and community labels - click here for full details

Apache Nifi PutS3Object integrity check

avatar
Frequent Visitor

Hello, 

Does NiFi’s PutS3Object processor perform an MD5 integrity check for the files uploaded to Amazon S3?

Thank you

Cristian

1 ACCEPTED SOLUTION

avatar
Master Collaborator

Hello @criki

No, PutS3Object itself does not perform MD5 integrity check. 
This can be confirmed on the processor source code where we do not have any checksum method: 
https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/... 

On this link you can see how the SDK can configure the MD5 checksum, which is not part of the processor code: 
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html 

What you can do is, in your Flow, add a hash and then review it. 
First with CryptographicHashContent create the hash, then do the PutS3Object. 
After that, you can confirm it is fine do FetchS3Object then CryptographicHashContent and compare the hash with RouteOnAttribute. 


Regards,
Andrés Fallas
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs-up button.

View solution in original post

1 REPLY 1

avatar
Master Collaborator

Hello @criki

No, PutS3Object itself does not perform MD5 integrity check. 
This can be confirmed on the processor source code where we do not have any checksum method: 
https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/... 

On this link you can see how the SDK can configure the MD5 checksum, which is not part of the processor code: 
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html 

What you can do is, in your Flow, add a hash and then review it. 
First with CryptographicHashContent create the hash, then do the PutS3Object. 
After that, you can confirm it is fine do FetchS3Object then CryptographicHashContent and compare the hash with RouteOnAttribute. 


Regards,
Andrés Fallas
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs-up button.