Hive + Hiveserver2 + Tez

[#1]

Hi,

I was going through the documentation and it was mentioned that as a best practice for different types of workloads multiple hiveserver2 instances are preferred.

I currently have a 9 node setup with 2 as (NN + SNN) and 7 data nodes. Currently the hive client is installed on one node and hiveserver2 is also installed in the same node. I have followed the documentation and got the hive client and hiveserver2 installed on another node. My question is

1. If I want to use separate tez configurations for hive is changing hive-site.xml sufficient?

2. How to setup tez on this new node?

3. How will hive or any other application know about tez that has been newly setup on this node? I am trying to basically understand how an application (hive, pig etc) is aware of tez.

Thanks,

Jayadeep

[#2 · Accepted solution]

@Jayadeep Jayaraman

Please see my inline comments.

1. If I want to use separate tez configurations for hive is changing hive-site.xml sufficient?

take advantage to hive session level property settings or use --hiveconf to set the session level param to override the default hive/tez params

2. How to setup tez on this new node?

install tez-client on this new node

3. How will hive or any other application know about tez that has been newly setup on this node? I am trying to basically understand how an application (hive, pig etc) is aware of tez.

tez.lib.uris point to tez jars on hdfs which tells hive client to use tez library at runtime. to see this location you can use set tez.lib.uris; from your client