Support Questions
Find answers, ask questions, and share your expertise

HIVE LLAP: managing concurrent queries

Solved Go to solution
Highlighted

HIVE LLAP: managing concurrent queries

Explorer

When running a large Hive query that takes up the whole cluster (all LLAP executors are busy), simultaneously trying to run even a simplest query takes a lot of time, probably because the process is waiting for a free executor to work with.

Is it possible to restrict/manage the queues in a similar manner as does YARN, i.e., so that there would always be at least one free executor for a quick query, or that the tasks from concurrent queries would be pushed in front of the line to guarantee some minimal amount of exposure to the executors these tasks would get?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: HIVE LLAP: managing concurrent queries

Contributor

The functionality like that is available in Apache Hive 3.0 (bugfixes in 3.1), as well as HDP 3.0/1; see the "workload management" feature. https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/hive-workload/content/hive_workload_managem... and https://issues.apache.org/jira/browse/HIVE-17481

View solution in original post

2 REPLIES 2

Re: HIVE LLAP: managing concurrent queries

Contributor

The functionality like that is available in Apache Hive 3.0 (bugfixes in 3.1), as well as HDP 3.0/1; see the "workload management" feature. https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.0/hive-workload/content/hive_workload_managem... and https://issues.apache.org/jira/browse/HIVE-17481

View solution in original post

Highlighted

Re: HIVE LLAP: managing concurrent queries

Explorer

Oh, that's good to hear, looking forward to upgrade then! Thank you for the heads up!