Support Questions
Find answers, ask questions, and share your expertise

Any guidelines for disk space allocation for tez job Spills?

Solved Go to solution
Highlighted

Any guidelines for disk space allocation for tez job Spills?

What is the disk space requirement for job spills. Is there any guideline for the same? Or is it just Job dependent - if so, how to determine the same?

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Any guidelines for disk space allocation for tez job Spills?

Expert Contributor

In general I configure disk space allocation for tez job spills the same way as Yarn intermediate data.

Please find here some discussions regarding how to configure it:

http://community.hortonworks.com/questions/2230/recommended-size-for-yarnnodemanagerresourcelocal.ht...

http://community.hortonworks.com/questions/1405/can-you-please-advise-about-how-best-to-use-this-s.h...

View solution in original post

1 REPLY 1
Highlighted

Re: Any guidelines for disk space allocation for tez job Spills?

Expert Contributor

In general I configure disk space allocation for tez job spills the same way as Yarn intermediate data.

Please find here some discussions regarding how to configure it:

http://community.hortonworks.com/questions/2230/recommended-size-for-yarnnodemanagerresourcelocal.ht...

http://community.hortonworks.com/questions/1405/can-you-please-advise-about-how-best-to-use-this-s.h...

View solution in original post