Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive CLI issue when killing with Ctrl+c

Solved Go to solution
Highlighted

Hive CLI issue when killing with Ctrl+c

Guru

I am running HPD 2.4.0.0 installed with Hive 1.2. If I exit the Hive CLI using Ctrl+c, then try to run Hive again immediately, it takes about 10 minutes for the Hive prompt to appear. It just hangs right here:

root@node1 bin # hive
WARNING: Use "yarn jar" to launch YARN applications.
Logging initialized using configuration in file:/etc/hive/conf/hive-log4j.properties

Any thoughts on why this may be occurring?

BTW, if I exit Hive using the "exit;" command, everything is fine and I can startup Hive again right away with no issues.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Rising Star

@Rich Raposa When you start hive cli, it starts a yarn application, which can be viewed via resource manager UI.

When you 'exit', you finish the application. And when you starts hive cli again, it starts another yarn application. But when you do Ctrl+c, that does not finish the application although that application eventually times out. While that previous application still hangs around and when you start CLI again anyway, it try to start a new app, but there is probably not enough capacity because of the previous application. When that previous application eventually times out, a new application start running and that's when you can use Hive CLI again.

View solution in original post

7 REPLIES 7
Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Rising Star

@Rich Raposa When you start hive cli, it starts a yarn application, which can be viewed via resource manager UI.

When you 'exit', you finish the application. And when you starts hive cli again, it starts another yarn application. But when you do Ctrl+c, that does not finish the application although that application eventually times out. While that previous application still hangs around and when you start CLI again anyway, it try to start a new app, but there is probably not enough capacity because of the previous application. When that previous application eventually times out, a new application start running and that's when you can use Hive CLI again.

View solution in original post

Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Guru

Thanks - this is exactly what is happening. FYI: I think it also affecting Pig. Is there a way to set the timeout to be a much shorter value?

Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Super Guru

@Rich Raposa - In addition to what @Takahiko Saito said, you can verify this thing by running hive terminal as:

hive --hiveconf hive.execution.engine=mr

Do ctrl+c

It should not take time to come out as we are using "mr" execution engine.

Hope this information helps.

Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Explorer

I too got same issue and resolved by doing hive --hiveconf hive.execution.engine=mr . But, how to resolve it if we want to use Tez instead of mr ?

Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Expert Contributor

@rich

as given by @Kuldeep Kulkarni and @Takahiko Saito it launches an YARN appllication when the default execution engine of HIVE is TEZ, and starts waiting for the resources if the Queue is fully occupied and no more resources available.

@gvenkatesh if you want to continue with TEZ we can do the below to work around the issue (only workaround as it is related to resource limitation of the YARN)

1) Increase the MAX queue capacity

2) Increase the User limitfactor to be more than 1

3) we can reduce the minimum container size 512MB with that we will have an option to launch more containers (note: this needs to be considered based on cluster usage)

Highlighted

Re: Hive CLI issue when killing with Ctrl+c

Explorer

Thank you.

Re: Hive CLI issue when killing with Ctrl+c

Contributor

One hack would be to kill the yarn application for a faster resource reclaim, retrieving the appid from RM UI

yarn application -kill <appid>

Don't have an account?
Coming from Hortonworks? Activate your account here