Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

FetchS3Object - Failed to retrieve S3 object error - NiFi

avatar
Contributor

Hello

 

I am having an issue retrieving bucket contents into NiFi using the FetchS3Object processor. I have configured the ListS3 processor to pull in a json array containing all the bucket information, I'm happy that's working ok... example output:

Griggsy_3-1646990794189.png

 

I have configured the FetchS3Object processor with the same Accesskey, Secret etc but I get the following error, I haven't been able to find much online ref this error, can anyone see where I'm going wrong?

Griggsy_2-1646990708726.png

FetchS3Object Config:

Griggsy_4-1646990897739.png

 

Any assistance would be greatly appreciated, I've seen a lot of questions asked about these two processors but haven't found anything about this particular error.

 

Cheers

Andy

1 ACCEPTED SOLUTION

avatar
Contributor

Hello @Azhar_Shaikh 

 

Thanks for the reply, as it turns out it wasn't a service account problem. We found that the ListS3's output included a 'key' field, and this is what was required in The FetchS3Object processor for 'Object Key'. 

 

So the fix I applied was to split the json into individual records (SplitJson), then pull the keys out as attributes (EvaluateJsonPath) then input ${key} into the FetchS3 processor.. worked a treat.

Griggsy_0-1647358844035.png

 

View solution in original post

2 REPLIES 2

avatar
Master Collaborator

Hello @Griggsy 

 

I can see from the error (Service:Amazon S3 Error code :403, Access Denied) 

 

The error code 403 means to be an authorization issue from cloud provider end.  If you are using any service account with the processor, can you please check and confirm if you the service account have access to the bucket that you have configured in Nifi ? 

 

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

 

Thanks,

Azhar

avatar
Contributor

Hello @Azhar_Shaikh 

 

Thanks for the reply, as it turns out it wasn't a service account problem. We found that the ListS3's output included a 'key' field, and this is what was required in The FetchS3Object processor for 'Object Key'. 

 

So the fix I applied was to split the json into individual records (SplitJson), then pull the keys out as attributes (EvaluateJsonPath) then input ${key} into the FetchS3 processor.. worked a treat.

Griggsy_0-1647358844035.png