Support Questions

Find answers, ask questions, and share your expertise

Hive on Spark

avatar
Explorer

Do Hortonworks plan supporting Hive on Spark ? or Is Hive on Spark a cloudera direction - and Hive LLAP Hortonworks' ?

1 ACCEPTED SOLUTION

avatar

@Yasir Faiz

Hortonworks fully supports Spark but we do not support Hive on Spark. There are couple of reasons for this:

1. Performance: Hive on Tez showed a 50x or greater improvement than Hive on MR, while Hive on Spark only showed a 3x improvement over Hive on MR.

2. Scale: Hive on Tez has been proven to scale at data sets well exceeding what is capable by Hive on Spark.

Here is a presentation discussing some of the differences: http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final

With LLAP, Hive speed is increased even further but LLAP is not a replacement for Tez. In fact, LLAP still leverages the Tez engine.

Hope this helps!

View solution in original post

7 REPLIES 7

avatar

@Yasir Faiz

Hortonworks fully supports Spark but we do not support Hive on Spark. There are couple of reasons for this:

1. Performance: Hive on Tez showed a 50x or greater improvement than Hive on MR, while Hive on Spark only showed a 3x improvement over Hive on MR.

2. Scale: Hive on Tez has been proven to scale at data sets well exceeding what is capable by Hive on Spark.

Here is a presentation discussing some of the differences: http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final

With LLAP, Hive speed is increased even further but LLAP is not a replacement for Tez. In fact, LLAP still leverages the Tez engine.

Hope this helps!

avatar
Contributor

Thank you for valuable information.

avatar
Explorer

Thanks @Scott Shaw. It does help. Do we know when Hortonworks would fully support LLAP ? As of 2.5.3 which i assume is latest - its still in tech preview mode ?

avatar
Explorer

Thanks @Scott Shaw. This helps. Do you know when Hortonworks would fully support LLAP in production ? I see as of 2.5.3 its still in tech preview ? Is there any advantage in terms of hive from moving from HDP 2.3 to HD 2.5...as i see both are using Hive 1.2.1.

avatar

@Yasir Faiz

I believe LLAP will be GA in 2.6 which is due out sometime in Q2. The advantage of HDP 2.5 would be to test LLAP and Hive 2.0. Also, HDP 2.5 also includes a number fixes to Hive 1.2.1 1.2.1 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/fixed_issues.html

avatar
Explorer

Hello, I just want to know do Hortonworks plan supporting Hive on Spark now?
Thanks very much!

@Scott Shaw

,

Hello,I just want to know do Hortonworks support Hive on Spark now?Tanks very much! @Scott Shaw

avatar
Contributor

I would also like to know this. Is Hive on Spark currently supported by HortonWorks? Since which version?

 

Thanks in advance,

David Resende