question Re: Can I limit the number of containers allocated by Tez? in Archives of Support Questions (Read Only)

Can I limit the number of containers allocated by Tez?

diablo2 — Mon, 31 Oct 2016 19:50:13 GMT

I found sometime, in our cluster, only one query from Tez can used up all resources. Can I limit the containers for Tez query so that there are free resource kept for other sessions?

Re: Can I limit the number of containers allocated by Tez?

KuldeepK — Mon, 31 Oct 2016 22:19:42 GMT

@Huahua Wei

Please check your capacity schedule configuration, on the queue where TEZ container is occupying all the capacity. Also if you are using only default queue then please check size of your tez container, modify it accordingly. If you are using default queue then I would recommend you to configure multiple queues.

Please refer below doc for more details

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.4.3/bk_installing_manually_book/content/ref-475bae8f-9da0-4bcb-955c-e9722c4c536a.1.html

Re: Can I limit the number of containers allocated by Tez?

diablo2 — Tue, 01 Nov 2016 08:41:26 GMT

scheduler-config-1477964292460.xmlKuldeep, Thanks for your quick response.

Firstly, tez used default queue, Now I created new which named squeue, but I don't know how to set the value for "User Limit Factor" and "MInimum User Limit" .

Re: Can I limit the number of containers allocated by Tez?

zyang — Thu, 03 Nov 2016 11:52:34 GMT

You can set hive/tez to use a specified queue. For tez, the configuration is 'tez.queu.name'. Regarding to queue capacity settings, please refer to this link.