Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Load balancing HiveServer2 Over Knox

Solved Go to solution
Highlighted

Load balancing HiveServer2 Over Knox

Expert Contributor

Hi,

Using https://knox.apache.org/books/knox-0-9-0/user-guide.html, I have configured Knox topology for Hive Server2 High Availability.

I also noticed Dynamic Service Discovery Through ZooKeeper in documentation.

I see that all the queries/connections happen though only one of the HiveServer2, now if this HS2 instances down, I notice that connections/queries happen through another instance of HS2.

My question is : In the busy cluster, when we have multiple HS2 servers installed, is it possible to load balance (possibly round robin) so that one server does not get overloaded? If yes, how?

Regards,

SS

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Load balancing HiveServer2 Over Knox

Expert Contributor

Your Network team can create a DNS Name and can add both the hiverserver2 IPs under this DNS in either round robin or load-balance mode. Then, you can use the DNS name and 10000 port in the Knox configuration to submit hive jobs. It should provide what you are looking for. Please let me know.

View solution in original post

4 REPLIES 4
Highlighted

Re: Load balancing HiveServer2 Over Knox

Expert Contributor

Your Network team can create a DNS Name and can add both the hiverserver2 IPs under this DNS in either round robin or load-balance mode. Then, you can use the DNS name and 10000 port in the Knox configuration to submit hive jobs. It should provide what you are looking for. Please let me know.

View solution in original post

Highlighted

Re: Load balancing HiveServer2 Over Knox

Expert Contributor

Thank you @Manish Gupta,

This is something we can try by having a proxy configured to have load balancing for HS2. Also would like to understand are they changes to be made for Zookeeper ?

Is there any other way by which, without using non HDP component or external network changes, we can achieve load balancing of HS2.

Regards,

Re: Load balancing HiveServer2 Over Knox

Expert Contributor

@Smart Solutions

No changes would be required for ZooKeeper. Unfortunatley, load-balancing of Hive Server2 feature is not available in HDP currently. Hopefully in future release.

As always, if you find this post helpful, don't forget to "accept" answer.

Highlighted

Re: Load balancing HiveServer2 Over Knox

Expert Contributor

Thanks @Manish Gupta for clarification. This is helpful.

Cheers.

Don't have an account?
Coming from Hortonworks? Activate your account here