Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Inconsistent time taken for the same query when run multiple times. ( Hive on Tez)

Inconsistent time taken for the same query when run multiple times. ( Hive on Tez)

Expert Contributor

Hi all,

When I run this query I get totally different time taken when I run on the same cluster. (This is Hive on Tez)

select * from database.tabeName limit 10;

The time taken to run this ranges from 2 seconds to 10 minutes!!

If sometimes it take 2 mins and next time it takes 2mins and 20 seconds it is still ok. But from 2 seconds to 10 mins.

What could be the possible reasons?

How can we make sure that they take similar time ?

Regards,

SS

2 REPLIES 2
Highlighted

Re: Inconsistent time taken for the same query when run multiple times. ( Hive on Tez)

New Contributor

Hi,

Select * from table does not really have to run any job. It just pulls data from HDFS and presents to you. It could be the resources on the edge node, or you may be pulling data from a view which runs a query underneath.

Can you pls answer the following questions:

1. are you using beeline ? If yes, hiveserver2 could be busy at times

2. Are you querying a table or view

3. what is the amount of memory on edge node ?

Re: Inconsistent time taken for the same query when run multiple times. ( Hive on Tez)

Expert Contributor

Hi @anatva :

It does spawn a Tez Job :

> select * from DATABASENAME.TABLE_NAME limit 10;
INFO  : Session is already open
INFO  : Dag name: select * from DATABASENAME.TAB...10(Stage-1)
INFO  : Status: Running (Executing on YARN cluster with App id application_1496614688621_2617)


--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED     33         33        0        0       0       0
Reducer 2 ......   SUCCEEDED    227        227        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 02/02  [==========================>>] 100%  ELAPSED TIME: 17.63 s
--------------------------------------------------------------------------------

Here are the answers.

1. are you using beeline ? If yes, hiveserver2 could be busy at times:

Yes and other tools which connect using JDBC using Knox.

2. Are you querying a table or view:

View

3. what is the amount of memory on edge node ?

256 GB