Support Questions

Find answers, ask questions, and share your expertise

Is there a way to run multiple HiveServer2 Interactive instanmces in the cluster and manage it through Ambari?

avatar
Contributor
 
1 ACCEPTED SOLUTION

avatar
Super Guru

@Leonid Fedotov

LLAP is still in Tech Preview and having more than one HiveServer2 Interactive is not currently supported. HiveServer2 Interactive service is deployed to the host that you want to run it on. As-is, it is a single point of failure and it supports a single queue. Let's remember that a Tech Preview feature is meant to demonstrate the concept, but it usually falls short on enterprise level features like HA, Security, Resource Management. Those are to be added.

It is a long shot to setup outside of Ambari. There are no documents to document such approach. You would have to create multiple HiveServer2 instances and point HiveServer2 interactive to those. I would not go that path. I would wait for Ambari 2.5.0 to have two instances.

If any response helped, please vote/accept best answer.

+++

It is likely that others may be confusing HiveServer2 and HiveServer2 Interactive services and it is important to understand the difference. I started to respond your question before you and @jss had another pass at it. I will still keep the content below for other HCC users.

HiveServer1 is a server client model service, which allow users to connect using Hive CLI interface and using thrift client. It support for remote client connection but only one client can connect at a time. It does not provide session management support and because of thrift API it does not provide concurrency control.

HiveServer 2 is also a client and server model and it allows to connect many different clients like thrift. HiveServer2 gives multi-client support where many clients can connect at the same time. Authentication is much better using kerberos. It provides support for JDBC and ODBC driver connection. Beeline CLI is used for connecting to HiveServer2.

Interactive HiveServer2has been around for a while and it does not have anything to do with Hive 2. The only thing that got better is that multiple interactive HiveServer2 instances can be created via Ambari UI using recent versions of HDP, I believe post HDP 2.2. In HDP 2.2 and earlier, interactive queues can be set up at the command-line. In HDP 2.3 and later, you can use Ambari (a GUI) to set up interactive queues.

Multiple HiveServer2 instances can be used for:

  • Load-balancing and high availability using Zookeeper
  • Running multiple applications with different settings

Because HiveServer2 uses its own settings file, using one for ETL operations and another for interactive queries is a common practice. All HiveServer2 instances can share the same MetastoreDB. Consequently, setting up multiple HiveServer2 instances that have embedded metastores is a simple operation.

HiveServer2 Interactive

This is still HiveServer2 instance, however, it is dedicated to HiveServer2 Interactive service. Check this reference: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_hive-performance-tuning/content/ch_hive_..., section 8.

View solution in original post

6 REPLIES 6

avatar

@ Leonid Fedotov

Yes, you can add multiple HiveServer2. Below link talks about how you can install another HiveServer2.

http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.0/bk_Ambari_Users_Guide/content/_adding_a_hive...

avatar
Contributor

@jss have you read question before answering?

I clearly asking about INTERACTIVE HiveServer2, whichis part of the Hive 2.1 which is in tech preview on HDP 2.5

Your answer is about good old boring Hive 1 HiveServer2

avatar

@Leonid Fedotov

Sorry i missed the "interactive' word there.

I see a JIRA for allowing multiple HSI ( proposed for ambari 2.5)

Allow two instances of Hive Server Interactive: https://issues.apache.org/jira/browse/AMBARI-18917

avatar
Contributor

So, the answer is "no" for Ambari support. Expected...

What about non-ambani configuration? Is it possible? is there any docs on this?

avatar
Super Guru

@Leonid Fedotov

LLAP is still in Tech Preview and having more than one HiveServer2 Interactive is not currently supported. HiveServer2 Interactive service is deployed to the host that you want to run it on. As-is, it is a single point of failure and it supports a single queue. Let's remember that a Tech Preview feature is meant to demonstrate the concept, but it usually falls short on enterprise level features like HA, Security, Resource Management. Those are to be added.

It is a long shot to setup outside of Ambari. There are no documents to document such approach. You would have to create multiple HiveServer2 instances and point HiveServer2 interactive to those. I would not go that path. I would wait for Ambari 2.5.0 to have two instances.

If any response helped, please vote/accept best answer.

+++

It is likely that others may be confusing HiveServer2 and HiveServer2 Interactive services and it is important to understand the difference. I started to respond your question before you and @jss had another pass at it. I will still keep the content below for other HCC users.

HiveServer1 is a server client model service, which allow users to connect using Hive CLI interface and using thrift client. It support for remote client connection but only one client can connect at a time. It does not provide session management support and because of thrift API it does not provide concurrency control.

HiveServer 2 is also a client and server model and it allows to connect many different clients like thrift. HiveServer2 gives multi-client support where many clients can connect at the same time. Authentication is much better using kerberos. It provides support for JDBC and ODBC driver connection. Beeline CLI is used for connecting to HiveServer2.

Interactive HiveServer2has been around for a while and it does not have anything to do with Hive 2. The only thing that got better is that multiple interactive HiveServer2 instances can be created via Ambari UI using recent versions of HDP, I believe post HDP 2.2. In HDP 2.2 and earlier, interactive queues can be set up at the command-line. In HDP 2.3 and later, you can use Ambari (a GUI) to set up interactive queues.

Multiple HiveServer2 instances can be used for:

  • Load-balancing and high availability using Zookeeper
  • Running multiple applications with different settings

Because HiveServer2 uses its own settings file, using one for ETL operations and another for interactive queries is a common practice. All HiveServer2 instances can share the same MetastoreDB. Consequently, setting up multiple HiveServer2 instances that have embedded metastores is a simple operation.

HiveServer2 Interactive

This is still HiveServer2 instance, however, it is dedicated to HiveServer2 Interactive service. Check this reference: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.0/bk_hive-performance-tuning/content/ch_hive_..., section 8.

avatar
Contributor

Now in 2.6, I understand Hive LLAP is production ready but it still only allows a single HSI which becomes a SPOF There was a patch contributed but it was never accepted: https://issues.apache.org/jira/browse/AMBARI-18917 Do you have any information about why something as important yet apparently trivial hasn't been addressed?