Created 02-06-2025 07:58 PM
Hello,
I'm trying to upload file to S3 compatible IBM Cloud Object Storage by PutS3Object processor.
It fails with the attached error only when target ICOS bucket is configured to protect object.
When we upload file with s3cmd, it works.
Is there any solution for this problem?
Thanks,
2025-02-07 10:17:00,892 ERROR [Timer-Driven Process Thread-6] o.a.nifi.processors.aws.s3.PutS3Object PutS3Object[id=b4359d36-0193-1000-4303-1924a76d8f5a] Failed to put StandardFlowFileRecord[uuid=3f891ca3-d3b9-4ffa-b446-306386f637db,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1738891020007-580, container=default, section=580], offset=0, length=894211],offset=0,name=89e42cbe-172f-4882-8b9e-695b27c7a6e5,size=894211] to Amazon S3 due to com.amazonaws.services.s3.model.AmazonS3Exception: Missing required content hash for this request: Content-MD5 or x-amz-content-sha256 (Service: Amazon S3; Status Code: 400; Error Code: MissingDigest; Request ID: db2b96e4-d540-4ca9-9dd1-0605ad4b8658; S3 Extended Request ID: null; Proxy: null), S3 Extended Request ID: null
com.amazonaws.services.s3.model.AmazonS3Exception: Missing required content hash for this request: Content-MD5 or x-amz-content-sha256 (Service: Amazon S3; Status Code: 400; Error Code: MissingDigest; Request ID: db2b96e4-d540-4ca9-9dd1-0605ad4b8658; S3 Extended Request ID: null; Proxy: null)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1912)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1450)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1419)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1183)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:838)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:805)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:779)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:735)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:581)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:559)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5590)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5537)
at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:428)
at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:6671)
at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1913)
at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1873)
at org.apache.nifi.processors.aws.s3.PutS3Object.onTrigger(PutS3Object.java:615)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1274)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:244)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:358)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
at java.base/java.lang.Thread.run(Thread.java:1583)
Created 02-06-2025 11:00 PM
Additional information.
It looks file upload fails when retention policy is enabled on the target ICOS bucket.
Thanks,