question Re: Is there a way to prevent the start of the Tez AM from the Hive CLI? in Archives of Support Questions (Read Only)

Is there a way to prevent the start of the Tez AM from the Hive CLI?

kkane — Sat, 05 Dec 2015 00:23:13 GMT

The desired end result is that I want no YARN containers automatically started when I enter the Hive CLI and the engine default is Tez.

hive.prewarm prevents default work containers but I need to stop the Tez AM from launching.

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

nsabharwal — Sat, 05 Dec 2015 00:24:07 GMT

@gopal

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

kkane — Sat, 05 Dec 2015 00:28:20 GMT

Looks like the answer is no:

https://issues.apache.org/jira/browse/HIVE-9602

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

nsabharwal — Sat, 05 Dec 2015 00:29:41 GMT

@sseth

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

kkane — Sat, 05 Dec 2015 00:39:12 GMT

For anyone that is interested in why I want this:

People use Oozie Shell actions to orchestrate work because they are familiar/comfortable with bash.

When using the Hive CLI from a YARN container context and secure cluster the engine specification must be done in an early stage.

For the Hive CLI this means users must change their existing HQL from

 hive -e 'set tez.credentials.path=${HADOOP_TOKEN_FILE_LOCATION}' 

to 

--hiveconf tez.credentials.path=${HADOOP_TOKEN_FILE_LOCATION}

For Sqoop, there is no way to affect this change from configuration and the user must ship a custom hive-site.xml in the distributed cache of the Oozie-shell.  This hive-site.xml must set the engine to MR

If this is not done, Hive and Sqoop will launch a Tez AM that may never even be used but which fails due to missing the delegation token.

These changes are small from from this explanation but rather large for organizations that already have established HQL and the cluster upgrade includes a default engine change from MR to Tez.

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

deepesh1 — Sat, 05 Dec 2015 01:03:12 GMT

A workaround for this could be launching the hive cli in the following manner:

hive -hiveconf hive.execution.engine=mr

But this would mean that if you want to run any queries in tez you would need to run "set hive.execution.engine=tez;" before running your queries.

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

bsaini — Sat, 05 Dec 2015 02:08:31 GMT

Can you help understand the scenario when this is needed? So the Hive shell is executed but wait until a query is executed for creating AM.. this means there are situations where Hive shell is executed and then exited without executing the query? Wont this be an exception scenario or in your case this is so frequent / regular that a workaround is required. I am sorry, just trying to understand when will such a configuration be needed..

Re: Is there a way to prevent the start of the Tez AM from the Hive CLI?

aervits — Wed, 03 Feb 2016 23:50:03 GMT

@kkane are you still having issues with this? Can you accept best answer or provide your own solution?