Dedicate hiveservers to different group of users or use case using ambari config groups

[#1]

Use Cases:

• Seperate out hiveservers for batch jobs which can ensure user jobs or too many connections from users dont effect batch workloads or one groups workloads dont effect other group workloads.
• Secure hive connections through Knox which only works with http mode for hiveservers2.
• Custom configurations for different workloads such as different engine's MR or tez or anything else.

Method:

Use ambari config groups to customize the default hive.server2.zookeeper.namespace to something else other than default value “hiveserver2”

Reference: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_hadoop-high-availability/content/ha-hs2-s...

http://docs.hortonworks.com/HDPDocuments/Ambari-2.5.0.3/bk_ambari-operations/content/using_host_conf...

Caveat: The beeline connection string will be different from the default one. The new one needs different value for the zooKeeperNamespace which can cause confusion.

Example:

I will be using Ambari 2.4.2 with HDP 2.3.2 but should work with anything higher.

In this cluster we have two hosts wg-pm1 and wg-pm3 that have hiveserver2’s installed and running with default configuration.

Below, I will update the configurations such that hiveserver2 on wg-pm1 will run in hive.server2.zookeeper.namespace hiveserver2-batch.

Click on the override button and enter a name for ambari hive configuration group.

One host can only be part of one ambari config group so choose the name wisely.

If you already have existing config group you would like to use, you may use that.

I choose to use new hive configuration group with name HiveServer2-batch

Click “Manage Hosts” to Update the hosts that are part of the new config group.

Then chose the hiveserver-batch and add the host and save

Now ambari should prompt to customize the hive.server2.zookeeper.namespace option

I chose to use hiveserver2-batch as the zookeeper namespace for hiveserver2 and click save.

Ambari should prompt to restart the affected hiveservers, after restart, dedicated hiveservers can be discovered only through the new zookeeper namespace.

My default beeline jdbc string is as below:

beeline -u "jdbc:hive2://wg-pm1.wg.com:2181,wg-pm2.wg.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2"

To connect to the dedicated hiveservers we will need to use customized value of hive.server2.zookeeper.namespace for the value of zooKeeperNamespace

beeline -u
"jdbc:hive2://wg-pm1.wg.com:2181,wg-pm2.wg.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-batch"