Support Questions
Find answers, ask questions, and share your expertise

How to configure multiple S3 end-points in a multi-tenanted cluster?

I have a multi-tenanted HDP2.3 cluster. It has been configured with an S3 end-point in custom hdfs-site.xml. Is it possible to add another S3 end-point for another tenant? If so, what should be the property name?

Thanks in Advance.

3 REPLIES 3

Re: How to configure multiple S3 end-points in a multi-tenanted cluster?

Hi @Phoncy Joseph In HDP 2.6.1 you can set per-bucket properties to authenticate with multiple buckets. See https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_cloud-data-access/content/s3-auth-per-bu....

However, I believe that this was introduced in HDP 2.5 or 2.6 so it is most likely not available in HDP 2.3.

CC @stevel

Re: How to configure multiple S3 end-points in a multi-tenanted cluster?

New Contributor

I have using HDP 2.5 but when i configured multiple buckets after that doing LS it throwing access denied error 403.

Re: How to configure multiple S3 end-points in a multi-tenanted cluster?

@Phoncy Joseph To add to the previous answer:

If you upgrade to HDP 2.6.1, there is a new Ranger feature that allows you to control access to S3 buckets via Hive by different user. It's a new "URL" parameter available when creating a Hive policy: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.1/bk_security/content/hive_policy.html.

So once you add multiple S3 buckets you can control which user can read or write from which buckets.