question Re: Hive on Spark in Archives of Support Questions (Read Only)

Hive on Spark

jasir_faiz — Wed, 18 Jan 2017 23:19:44 GMT

Do Hortonworks plan supporting Hive on Spark ? or Is Hive on Spark a cloudera direction - and Hive LLAP Hortonworks' ?

Re: Hive on Spark

SQLShaw — Wed, 18 Jan 2017 23:34:49 GMT

Hortonworks fully supports Spark but we do not support Hive on Spark. There are couple of reasons for this:

1. Performance: Hive on Tez showed a 50x or greater improvement than Hive on MR, while Hive on Spark only showed a 3x improvement over Hive on MR.

2. Scale: Hive on Tez has been proven to scale at data sets well exceeding what is capable by Hive on Spark.

Here is a presentation discussing some of the differences: http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final

With LLAP, Hive speed is increased even further but LLAP is not a replacement for Tez. In fact, LLAP still leverages the Tez engine.

Hope this helps!

Re: Hive on Spark

jasir_faiz — Wed, 18 Jan 2017 23:41:10 GMT

Thanks @Scott Shaw. It does help. Do we know when Hortonworks would fully support LLAP ? As of 2.5.3 which i assume is latest - its still in tech preview mode ?

Re: Hive on Spark

jasir_faiz — Wed, 18 Jan 2017 23:43:30 GMT

Thanks @Scott Shaw. This helps. Do you know when Hortonworks would fully support LLAP in production ? I see as of 2.5.3 its still in tech preview ? Is there any advantage in terms of hive from moving from HDP 2.3 to HD 2.5...as i see both are using Hive 1.2.1.

Re: Hive on Spark

SQLShaw — Thu, 19 Jan 2017 00:08:22 GMT

@Yasir Faiz

I believe LLAP will be GA in 2.6 which is due out sometime in Q2. The advantage of HDP 2.5 would be to test LLAP and Hive 2.0. Also, HDP 2.5 also includes a number fixes to Hive 1.2.1 1.2.1 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/fixed_issues.html

Re: Hive on Spark

qiflyqi — Tue, 15 Jan 2019 15:20:31 GMT

Hello, I just want to know do Hortonworks plan supporting Hive on Spark now?
Thanks very much!

@Scott Shaw

Hello，I just want to know do Hortonworks support Hive on Spark now？Tanks very much! @Scott Shaw

Re: Hive on Spark

shashank_naresh — Sun, 05 May 2019 12:05:41 GMT

Thank you for valuable information.

Re: Hive on Spark

DavidR — Mon, 10 Feb 2020 17:37:59 GMT

I would also like to know this. Is Hive on Spark currently supported by HortonWorks? Since which version?

Thanks in advance,

David Resende