question Hive CLI issue when killing with Ctrl+c in Support Questions

Hive CLI issue when killing with Ctrl+c

rich1 — Wed, 22 Jun 2016 06:33:22 GMT

I am running HPD 2.4.0.0 installed with Hive 1.2. If I exit the Hive CLI using Ctrl+c, then try to run Hive again immediately, it takes about 10 minutes for the Hive prompt to appear. It just hangs right here:

root@node1 bin # hive
WARNING: Use "yarn jar" to launch YARN applications.
Logging initialized using configuration in file:/etc/hive/conf/hive-log4j.properties

Any thoughts on why this may be occurring?

BTW, if I exit Hive using the "exit;" command, everything is fine and I can startup Hive again right away with no issues.

Re: Hive CLI issue when killing with Ctrl+c

tsaito — Wed, 22 Jun 2016 06:43:47 GMT

@Rich Raposa When you start hive cli, it starts a yarn application, which can be viewed via resource manager UI.

When you 'exit', you finish the application. And when you starts hive cli again, it starts another yarn application. But when you do Ctrl+c, that does not finish the application although that application eventually times out. While that previous application still hangs around and when you start CLI again anyway, it try to start a new app, but there is probably not enough capacity because of the previous application. When that previous application eventually times out, a new application start running and that's when you can use Hive CLI again.

Re: Hive CLI issue when killing with Ctrl+c

KuldeepK — Wed, 22 Jun 2016 07:09:51 GMT

@Rich Raposa - In addition to what @Takahiko Saito said, you can verify this thing by running hive terminal as:

hive --hiveconf hive.execution.engine=mr

Do ctrl+c

It should not take time to come out as we are using "mr" execution engine.

Hope this information helps.

Re: Hive CLI issue when killing with Ctrl+c

rich1 — Wed, 22 Jun 2016 11:00:20 GMT

Thanks - this is exactly what is happening. FYI: I think it also affecting Pig. Is there a way to set the timeout to be a much shorter value?

Re: Hive CLI issue when killing with Ctrl+c

gandra — Wed, 02 Aug 2017 06:58:36 GMT

I too got same issue and resolved by doing hive --hiveconf hive.execution.engine=mr . But, how to resolve it if we want to use Tez instead of mr ?

Re: Hive CLI issue when killing with Ctrl+c

dheer_vijji_rag — Wed, 02 Aug 2017 14:02:33 GMT

@rich

as given by @Kuldeep Kulkarni and @Takahiko Saito it launches an YARN appllication when the default execution engine of HIVE is TEZ, and starts waiting for the resources if the Queue is fully occupied and no more resources available.

@gvenkatesh if you want to continue with TEZ we can do the below to work around the issue (only workaround as it is related to resource limitation of the YARN)

1) Increase the MAX queue capacity

2) Increase the User limitfactor to be more than 1

3) we can reduce the minimum container size 512MB with that we will have an option to launch more containers (note: this needs to be considered based on cluster usage)

Re: Hive CLI issue when killing with Ctrl+c

gandra — Wed, 02 Aug 2017 23:17:11 GMT

Thank you.

Re: Hive CLI issue when killing with Ctrl+c

asreekumar — Thu, 03 Aug 2017 01:10:54 GMT

One hack would be to kill the yarn application for a faster resource reclaim, retrieving the appid from RM UI

yarn application -kill <appid>