question Re: Hive on Spark vs Impala in Archives of Support Questions (Read Only)

Hive on Spark vs Impala

AlinaGHERMAN — Fri, 16 Sep 2022 10:07:33 GMT

Hello,

In CDH 5.6 there is Hive on Spark and Impala.

How should we choose between these 2 services? Are there any benchmarks that compare these 2 services?

Thank you! 🙂

Re: Hive on Spark vs Impala

jkestelyn — Mon, 07 Mar 2016 22:04:39 GMT

Hi Alina,

Although Hive-on-Spark will definitely provide improved performance over MR for batch processing applications (eg ETL), that performance is not going to approach the interactive "BI" experience provided by Impala.

Here's some recent Impala performance testing results:

http://blog.cloudera.com/blog/2016/02/new-sql-benchmarks-apache-impala-incubating-2-3-uniquely-delivers-analytic-database-performance/

Although Hive-on-Spark is not included, one would expect it to perform at levels similar to that of Hive-on-Tez (although having the added advantage of supporting consolidation onto the Spark API).

Re: Hive on Spark vs Impala

izhar — Mon, 18 Apr 2016 08:38:58 GMT

What is cloudera's take on usage for Impala vs Hive-on-Spark?

We would also like to know what are the long term implications of introducing Hive-on-Spark vs Impala.

It would be definitely very interesting to have a head-to-head comparison between Impala, Hive on Spark and Stinger for example. I wouldnt include sparkSQL in here because in my opinion sparkSQL serves a totally different purpose.

/Izhar

Re: Hive on Spark vs Impala

jkestelyn — Mon, 18 Apr 2016 18:17:38 GMT

Was there anything in my answers to these questions higher in the thread unclear?

Re: Hive on Spark vs Impala

Raj2cool16 — Mon, 16 May 2016 07:09:35 GMT

Can you please help me in this?

http://community.cloudera.com/t5/Advanced-Analytics-Apache-Spark/Pros-and-Cons-of-fetching-data-using-SparkSQL-vs-Hive-in-Spark/m-p/40855#U40855