Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

HDP on AWS using EFS mounts for yarn and hdfs

Solved Go to solution
Highlighted

HDP on AWS using EFS mounts for yarn and hdfs

New Contributor

Hello,

Has anyone tired using aws EFS mounts as yarn scratch and hdfs directories?

The data long term will sit in S3.

Please let me know your thoughts.

Thanks,

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HDP on AWS using EFS mounts for yarn and hdfs

Rising Star

Hi Mike yes, i have seen it once, the efs must be in the same Availability Zone ( if not, you may be charged for data transfer and have some perf degradation .. ) and i think the only downside is that it's network performance , it's less performant that having a local disk. i think it's ok if your use case permit , however if you are planning to use Hbase/Storm where latency is critical , i recommend you to do some benchmarking first.

1 REPLY 1

Re: HDP on AWS using EFS mounts for yarn and hdfs

Rising Star

Hi Mike yes, i have seen it once, the efs must be in the same Availability Zone ( if not, you may be charged for data transfer and have some perf degradation .. ) and i think the only downside is that it's network performance , it's less performant that having a local disk. i think it's ok if your use case permit , however if you are planning to use Hbase/Storm where latency is critical , i recommend you to do some benchmarking first.

Don't have an account?
Coming from Hortonworks? Activate your account here