Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Can we create queues under LLAP?

Highlighted

Can we create queues under LLAP?

New Contributor

I have a bunch of questions about Hive LLAP and its queues.

1. I have to run two types of queries on LLAP and want to set two different queues for each of them. I tried setting it up but Yarn resource manager failed to start saying it failed to initiate the queues.

2. How can I connect from beeline to Hive/LLAP on a specific queue?

3. How many concurrent users can LLAP have? I have a cluster with 8 data nodes and we currently set LLAP queue capacity to 90%. It is not letting me have more than 4 concurrent users. (Well I can have more than 4 but it is using 100% of the cluster which is bad. I thought it divides the resources equally among all sessions)

Thanks in advance.

3 REPLIES 3
Highlighted

Re: Can we create queues under LLAP?

Expert Contributor

No you can't, currently at least. LLAP supports only one queue instance as far as I know.

You may do a workaround by starting multiple LLAP sessions instead of sub-queuing. Here's the JIRA https://issues.apache.org/jira/browse/AMBARI-18917. I couldn't find a doc to do that, you may find with more search.

LLAP is very different from HS2, and the concept of sessions doesn't translate directly.

You can control concurrency only by setting the property "hive.server2.tez.sessions.per.default.queue"=desired sessions in hive-interactive-site.xml (if you're doing via ambari) or hive2/hive-site.xml (on disk) or modify directly via config section in hive.

Also num_llap_nodes in hive-interactive-env controls the number of nodes to run LLAP daemons, set this to 8 if you want it to run on all nodes or 7 (recommended).

It's also important to set optimal values of llap_queque %, llap_heap_size, cache memory, llap_num_executors, number of io threads for better performance. See below links for more info:-

https://community.hortonworks.com/questions/84636/llap-not-using-io-cache.html

https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.5/bk_hive-performance-tuning/content/ch_hive_...

Highlighted

Re: Can we create queues under LLAP?

Expert Contributor

@tsharma

what if we asked the questions another way, could you split a llap queue instance so that you could assign resource from the llap queue like capacity schedule allows you to do now?

Highlighted

Re: Can we create queues under LLAP?

Expert Contributor

I think you can't, I'm not really sure though.

If you can't, I think your solution would be to start multiple LLAP instances (probably you would get a different url for both) and hence submit jobs. Or you could increase sessions in your llap queue hive.server2.tez.sessions.per.default.queue This would give you some concurrency.

This is a useful link https://issues.apache.org/jira/browse/AMBARI-16720

Don't have an account?
Coming from Hortonworks? Activate your account here