question Re: Round Robin Queuing in Archives of Support Questions (Read Only)

Round Robin Queuing

sunile_manjee — Tue, 16 Feb 2016 11:52:31 GMT

I read in this tutorial http://hortonworks.com/hadoop-tutorial/supercharging-interactive-queries-hive-tez/ about

Multi-tenancy with HiveServer2

There could be contentions when multiple users run large queries simultaneously. Processing queries with many containers could lead to lower latency. For this, 3 controls could be put in place:

Container re-use timeout
Tez split wave tuning
Round Robin Queuing setup

What is Round Robin Queuing Setup within Hive/Tez? How would one go about the setup? No details were provided.

Re: Round Robin Queuing

aervits — Tue, 16 Feb 2016 19:19:10 GMT

@Sunile Manjeethis is an educated guess but we're talking about YARN queues with RoundRobin schedule implementation. So instead of capacity scheduler we would implement another scheduler algorithm. I don't think it's viable unless cluster only dedicated to hive it would be cool to try having two root queues supporting multiple schedulers. http://m.linuxjournal.com/content/how-yarn-changed-hadoop-job-scheduling

So with that idea in mind you world assign an available queue in round robin fashion to each user.

Re: Round Robin Queuing

nsabharwal — Tue, 16 Feb 2016 19:30:43 GMT

@Sunile Manjee

This is related to connection to HS2 in round robin when we have multiple HS2 running and we don't want to hit the same HS2 instance. Link

The whole concept is derived from basic Round Robin algorithm

This leverages the Zookeeper

Re: Round Robin Queuing

nsabharwal — Tue, 16 Feb 2016 19:31:58 GMT

@Sunile Manjee See this thread on HS2 and ZK relationship https://community.hortonworks.com/questions/1342/how-to-enable-zookeeper-discovery-for-hiveserver2.html