Hive on Spark

[#1]

Do Hortonworks plan supporting Hive on Spark ? or Is Hive on Spark a cloudera direction - and Hive LLAP Hortonworks' ?

[#2 · Accepted solution]

@Yasir Faiz

Hortonworks fully supports Spark but we do not support Hive on Spark. There are couple of reasons for this:

1. Performance: Hive on Tez showed a 50x or greater improvement than Hive on MR, while Hive on Spark only showed a 3x improvement over Hive on MR.

2. Scale: Hive on Tez has been proven to scale at data sets well exceeding what is capable by Hive on Spark.

Here is a presentation discussing some of the differences: http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final

With LLAP, Hive speed is increased even further but LLAP is not a replacement for Tez. In fact, LLAP still leverages the Tez engine.

Hope this helps!

[#3]

Thank you for valuable information.

[#4]

Thanks @Scott Shaw. It does help. Do we know when Hortonworks would fully support LLAP ? As of 2.5.3 which i assume is latest - its still in tech preview mode ?

[#5]

Thanks @Scott Shaw. This helps. Do you know when Hortonworks would fully support LLAP in production ? I see as of 2.5.3 its still in tech preview ? Is there any advantage in terms of hive from moving from HDP 2.3 to HD 2.5...as i see both are using Hive 1.2.1.

[#6]

@Yasir Faiz

I believe LLAP will be GA in 2.6 which is due out sometime in Q2. The advantage of HDP 2.5 would be to test LLAP and Hive 2.0. Also, HDP 2.5 also includes a number fixes to Hive 1.2.1 1.2.1 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/fixed_issues.html

[#7]

Hello, I just want to know do Hortonworks plan supporting Hive on Spark now?
Thanks very much!

@Scott Shaw

,

Hello，I just want to know do Hortonworks support Hive on Spark now？Tanks very much! @Scott Shaw

[#8]

I would also like to know this. Is Hive on Spark currently supported by HortonWorks? Since which version?

Thanks in advance,

David Resende