Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Why 2 versions of Hive and Spark in HDP 2.6.5

SOLVED Go to solution
Highlighted

Why 2 versions of Hive and Spark in HDP 2.6.5

New Contributor

I am planning to install HDP 2.6.5 and was checking the components and their versions. I noticed

  • Apache Hive 1.2.1
  • Apache Hive 2.1.0

and

  • Apache Spark 1.6.3
  • Apache Spark 2.3.0

Why are there two versions of the same component? Any specific reason?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Why 2 versions of Hive and Spark in HDP 2.6.5

@Nanda Kumar

There are major changes between both hive and spark versions. So you need to decide which to install. And is possible to install both in same cluster.

For hive the differences between 1.2.1 and 2.1 are big, in case you like to take a look please check here:

https://hortonworks.com/blog/announcing-apache-hive-2-1-25x-faster-queries-much/

For spark 1.6.x and 2.x also we have major differences as well, you can read some more here:

https://hortonworks.com/blog/welcome-apache-spark-2-0/

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

4 REPLIES 4

Re: Why 2 versions of Hive and Spark in HDP 2.6.5

@Nanda Kumar

There are major changes between both hive and spark versions. So you need to decide which to install. And is possible to install both in same cluster.

For hive the differences between 1.2.1 and 2.1 are big, in case you like to take a look please check here:

https://hortonworks.com/blog/announcing-apache-hive-2-1-25x-faster-queries-much/

For spark 1.6.x and 2.x also we have major differences as well, you can read some more here:

https://hortonworks.com/blog/welcome-apache-spark-2-0/

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

Re: Why 2 versions of Hive and Spark in HDP 2.6.5

@Nanda Kumar Yes, both are compatible and supported! You can install both in the same cluster and use both of them. If you are satisfied with the answer please remember to login and mark the answer as "Accepted" Thanks!

Re: Why 2 versions of Hive and Spark in HDP 2.6.5

Contributor

@Felix Albani , I'm currently on HDP 2.6.5 and would like to install or even preferably upgrade to Hive 2.1 . Do you happen to know any documentation on how to perform the install/upgrade? Unfortunately I can not see Hive 2.1 in the list of services that I can add.

Re: Why 2 versions of Hive and Spark in HDP 2.6.5

New Contributor

@Felix Albani. Thank you for the response. Are both the versions of both components compatible with the underlying components like Hadoop & HBase versions which are part of HDP 2.6.5 despite having huge changes between those versions?