Support Questions

Find answers, ask questions, and share your expertise

Using IAM Role within Hadoop-AWS module

avatar
New Contributor

I am using a library that internally uses the hadoop-aws module and uses IAM Assumed Roles to connect to S3.

However, the documentation mentions the following:

 

The S3A connector supports assumed roles for authentication with AWS. 
A full set of login credentials must be provided, 
which will be used to obtain the assumed role and refresh it regularly.

 

Now, to configure the authentication mechanism, there is property fs.s3a.assumed.role.credentials.provider with a few possible values, but all of the values are based on top of credentials like AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEY etc. However, AWS recommends to use IAM Roles instead of access keys.

 

 

Is there a way to configure this S3 adapter using IAM Roles so that I do not need to define the access keys?

3 REPLIES 3

avatar
Community Manager

@jeet23 Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our Hive experts @Shmoo and @mszurap  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
New Contributor

HI Diana,

I couldn't find a label for Hadoop, this question relates to Hadoop specifically.

avatar
Community Manager

@sandeepV2 Any insights here? Thanks!


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: