Created on 06-01-2023 06:50 AM - edited 06-01-2023 07:05 AM
I am using a library that internally uses the hadoop-aws module and uses IAM Assumed Roles to connect to S3.
However, the documentation mentions the following:
The S3A connector supports assumed roles for authentication with AWS. A full set of login credentials must be provided, which will be used to obtain the assumed role and refresh it regularly.
Now, to configure the authentication mechanism, there is property fs.s3a.assumed.role.credentials.provider with a few possible values, but all of the values are based on top of credentials like AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY etc. However, AWS recommends to use IAM Roles instead of access keys.
Is there a way to configure this S3 adapter using IAM Roles so that I do not need to define the access keys?
Created 06-01-2023 07:33 AM
@jeet23 Welcome to the Cloudera Community!
To help you get the best possible solution, I have tagged our Hive experts @Shmoo and @mszurap who may be able to assist you further.
Please keep us updated on your post, and we hope you find a satisfactory solution to your query.
Regards,
Diana Torres,Created 06-01-2023 08:18 AM
HI Diana,
I couldn't find a label for Hadoop, this question relates to Hadoop specifically.
Created 06-01-2023 12:49 PM
@sandeepV2 Any insights here? Thanks!
Regards,
Diana Torres,