Skip to main content
tono425
Silver ⭐️⭐️⭐️
February 7, 2025
Question

PutS3Object fails to upload file to S3 compatible storage with object lock setting

Hello,

I'm trying to upload file to S3 compatible IBM Cloud Object Storage by PutS3Object processor.
It fails with the attached error only when target ICOS bucket is configured to protect object.
When we upload file with s3cmd, it works.
Is there any solution for this problem?

Thanks,

 

2025-02-07 10:17:00,892 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.aws.s3.PutS3Object PutS3Object[id=b4359d36-0193-1000-4303-1924a76d8f5a] Failed to put StandardFlowFileRecord[uuid=3f891ca3-d3b9-4ffa-b446-306386f637db,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1738891020007-580, container=default, section=580], offset=0, length=894211],offset=0,name=89e42cbe-172f-4882-8b9e-695b27c7a6e5,size=894211] to Amazon S3 due to com.amazonaws.services.s3.model.AmazonS3Exception: Missing required content hash for this request: Content-MD5 or x-amz-content-sha256 (Service: Amazon S3; Status Code: 400; Error Code: MissingDigest; Request ID: db2b96e4-d540-4ca9-9dd1-0605ad4b8658; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null
com.amazonaws.services.s3.model.AmazonS3Exception: Missing required content hash for this request: Content-MD5 or x-amz-content-sha256 (Service: Amazon S3; Status Code: 400; Error Code: MissingDigest; Request ID: db2b96e4-d540-4ca9-9dd1-0605ad4b8658; S3 Extended Request ID: null; Proxy: null)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1912)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1450)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1419)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1183)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:838)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:805)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:779)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:735)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:581)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5590)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5537)
at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:428)
at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6671)
at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1913)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1873)
at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:615)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:244)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)

 

 

1 reply

tono425
tono425Author
Silver ⭐️⭐️⭐️
February 7, 2025

Additional information.

It looks file upload fails when retention policy is enabled on the target ICOS bucket.

Thanks,

Translate