Hive CLI issue when killing with Ctrl+c

[#1]

I am running HPD 2.4.0.0 installed with Hive 1.2. If I exit the Hive CLI using Ctrl+c, then try to run Hive again immediately, it takes about 10 minutes for the Hive prompt to appear. It just hangs right here:

root@node1 bin # hive
WARNING: Use "yarn jar" to launch YARN applications.
Logging initialized using configuration in file:/etc/hive/conf/hive-log4j.properties

Any thoughts on why this may be occurring?

BTW, if I exit Hive using the "exit;" command, everything is fine and I can startup Hive again right away with no issues.

[#2 · Accepted solution]

@Rich Raposa When you start hive cli, it starts a yarn application, which can be viewed via resource manager UI.

When you 'exit', you finish the application. And when you starts hive cli again, it starts another yarn application. But when you do Ctrl+c, that does not finish the application although that application eventually times out. While that previous application still hangs around and when you start CLI again anyway, it try to start a new app, but there is probably not enough capacity because of the previous application. When that previous application eventually times out, a new application start running and that's when you can use Hive CLI again.

[#3]

Thanks - this is exactly what is happening. FYI: I think it also affecting Pig. Is there a way to set the timeout to be a much shorter value?

[#4]

@Rich Raposa - In addition to what @Takahiko Saito said, you can verify this thing by running hive terminal as:

hive --hiveconf hive.execution.engine=mr

Do ctrl+c

It should not take time to come out as we are using "mr" execution engine.

Hope this information helps.

[#5]

I too got same issue and resolved by doing hive --hiveconf hive.execution.engine=mr . But, how to resolve it if we want to use Tez instead of mr ?

[#6]

@rich

as given by @Kuldeep Kulkarni and @Takahiko Saito it launches an YARN appllication when the default execution engine of HIVE is TEZ, and starts waiting for the resources if the Queue is fully occupied and no more resources available.

@gvenkatesh if you want to continue with TEZ we can do the below to work around the issue (only workaround as it is related to resource limitation of the YARN)

1) Increase the MAX queue capacity

2) Increase the User limitfactor to be more than 1

3) we can reduce the minimum container size 512MB with that we will have an option to launch more containers (note: this needs to be considered based on cluster usage)

[#7]

Thank you.

[#8]

One hack would be to kill the yarn application for a faster resource reclaim, retrieving the appid from RM UI

yarn application -kill <appid>