Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

hive select query with limit

avatar

I am trying to fetch results data stored in hive orc table , but when I run the query it gives results fast but When I run the with limit clause it is taking too much of time .

select * from tablename where year="2019" and month="jan-mar" ;

- Runs perfectly within 30-40secs

select * from tablename limit 10;

- Gets stuck

1 REPLY 1

avatar
Moderator

Hello @amol_08 ,

 

thank you for raising your question about why a hive select query with limit fails, while without limit isn't.

 

Can you please specify

  • the Hadoop distribution and the version you are using? E.g. CDH5.16, HDP3.1.
  • what is the platform you are using, please? E.g. Hive, HiveServer2, Hive LLAP?

I am asking these clarification questions to rule out any known issue you might hit.

 

For this general problem statement I would like to raise your attention to our Cloudera Documentation [1] that describes the same type of query of "SELECT * FROM <table_name> LIMIT 10;" that will cause all partitions of the target table loaded into memory if the table was partitioned resulting memory pressure and how to tackle this issue.

 

Please let us know if the referenced documentation addresses your enquiry by accepting this post as a solution.


Thank you:
Ferenc

 

[1] https://docs.cloudera.com/documentation/enterprise/latest/topics/admin_hive_tuning.html#hs2_identify...


Ferenc Erdelyi, Technical Solutions Manager

Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.

Learn more about the Cloudera Community: