Our Community is getting an upgrade! To get everything ready for the relaunch, we’ll be placing the site in read-only mode starting September 21st.
We really appreciate your understanding while we get things set up behind the scenes. Catch up on all the exciting details about the move here.
Need help or have questions? Drop us a line at [email protected]

Support Questions

Find answers, ask questions, and share your expertise
Announcements
Share your experience with Cloudera on G2 and get a $25 Amazon Gift card.
Hi, I'm CLEO! Something exciting is coming to the Community. Stay Tuned!

Apache Nifi PutS3Object integrity check

avatar
Frequent Visitor

Hello, 

Does NiFi’s PutS3Object processor perform an MD5 integrity check for the files uploaded to Amazon S3?

Thank you

Cristian

1 ACCEPTED SOLUTION

avatar
Master Collaborator

Hello @criki, 

No, PutS3Object itself does not perform MD5 integrity check. 
This can be confirmed on the processor source code where we do not have any checksum method: 
https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/... 

On this link you can see how the SDK can configure the MD5 checksum, which is not part of the processor code: 
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html 

What you can do is, in your Flow, add a hash and then review it. 
First with CryptographicHashContent create the hash, then do the PutS3Object. 
After that, you can confirm it is fine do FetchS3Object then CryptographicHashContent and compare the hash with RouteOnAttribute. 


Regards,
Andrés Fallas
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs-up button.

View solution in original post

1 REPLY 1

avatar
Master Collaborator

Hello @criki, 

No, PutS3Object itself does not perform MD5 integrity check. 
This can be confirmed on the processor source code where we do not have any checksum method: 
https://github.com/apache/nifi/blob/main/nifi-extension-bundles/nifi-aws-bundle/nifi-aws-processors/... 

On this link you can see how the SDK can configure the MD5 checksum, which is not part of the processor code: 
https://docs.aws.amazon.com/AmazonS3/latest/userguide/checking-object-integrity-upload.html 

What you can do is, in your Flow, add a hash and then review it. 
First with CryptographicHashContent create the hash, then do the PutS3Object. 
After that, you can confirm it is fine do FetchS3Object then CryptographicHashContent and compare the hash with RouteOnAttribute. 


Regards,
Andrés Fallas
--
Was your question answered? Please take some time to click on "Accept as Solution" below this post.
If you find a reply useful, say thanks by clicking on the thumbs-up button.