Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Hive on Spark

Solved Go to solution

Hive on Spark

New Contributor

Do Hortonworks plan supporting Hive on Spark ? or Is Hive on Spark a cloudera direction - and Hive LLAP Hortonworks' ?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Hive on Spark

@Yasir Faiz

Hortonworks fully supports Spark but we do not support Hive on Spark. There are couple of reasons for this:

1. Performance: Hive on Tez showed a 50x or greater improvement than Hive on MR, while Hive on Spark only showed a 3x improvement over Hive on MR.

2. Scale: Hive on Tez has been proven to scale at data sets well exceeding what is capable by Hive on Spark.

Here is a presentation discussing some of the differences: http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final

With LLAP, Hive speed is increased even further but LLAP is not a replacement for Tez. In fact, LLAP still leverages the Tez engine.

Hope this helps!

7 REPLIES 7

Re: Hive on Spark

@Yasir Faiz

Hortonworks fully supports Spark but we do not support Hive on Spark. There are couple of reasons for this:

1. Performance: Hive on Tez showed a 50x or greater improvement than Hive on MR, while Hive on Spark only showed a 3x improvement over Hive on MR.

2. Scale: Hive on Tez has been proven to scale at data sets well exceeding what is capable by Hive on Spark.

Here is a presentation discussing some of the differences: http://www.slideshare.net/hortonworks/hive-on-spark-is-blazing-fast-or-is-it-final

With LLAP, Hive speed is increased even further but LLAP is not a replacement for Tez. In fact, LLAP still leverages the Tez engine.

Hope this helps!

Re: Hive on Spark

Thank you for valuable information.

Re: Hive on Spark

New Contributor

Thanks @Scott Shaw. It does help. Do we know when Hortonworks would fully support LLAP ? As of 2.5.3 which i assume is latest - its still in tech preview mode ?

Re: Hive on Spark

New Contributor

Thanks @Scott Shaw. This helps. Do you know when Hortonworks would fully support LLAP in production ? I see as of 2.5.3 its still in tech preview ? Is there any advantage in terms of hive from moving from HDP 2.3 to HD 2.5...as i see both are using Hive 1.2.1.

Re: Hive on Spark

@Yasir Faiz

I believe LLAP will be GA in 2.6 which is due out sometime in Q2. The advantage of HDP 2.5 would be to test LLAP and Hive 2.0. Also, HDP 2.5 also includes a number fixes to Hive 1.2.1 1.2.1 http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.5.3/bk_release-notes/content/fixed_issues.html

Re: Hive on Spark

New Contributor

Hello, I just want to know do Hortonworks plan supporting Hive on Spark now?
Thanks very much!

@Scott Shaw

,

Hello,I just want to know do Hortonworks support Hive on Spark now?Tanks very much! @Scott Shaw

Re: Hive on Spark

Explorer

I would also like to know this. Is Hive on Spark currently supported by HortonWorks? Since which version?

 

Thanks in advance,

David Resende

Don't have an account?
Coming from Hortonworks? Activate your account here