Spark 1.41. installation

syedzeeshanahme — Wed, 16 Nov 2016 00:41:36 GMT

Hi we have a business requirement to install Spark 1.4.1 on our cluster, so I want to know the exact ambari repository and HDP version which I should be installing.

Previously we installed HDP-2.3.6.0-3796 and it came with SPARK 1.3.1.2 which was not desired and upgrading SPARK was a tedious task which resulted in Cluster Corruption.

So I need the exact HDP version which will have 1.4.1.

Re: Spark 1.41. installation

sshimpi — Wed, 16 Nov 2016 01:29:07 GMT

@Zeeshan Ahmed

You can install HDP 2.3.2 which comes with Apache Spark 1.4.1

Here are the release notes - https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.2/bk_HDP_RelNotes/content/ch_relnotes_v232.html

Ambari version - Ambari-2.1.2.1

Here is ambari repo - https://docs.hortonworks.com/HDPDocuments/Ambari-2.1.2.1/bk_Installing_HDP_AMB/content/_download_the_ambari_repo_lnx6.html

wget -nv http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.2.1/ambari.repo -O /etc/yum.repos.d/ambari.repo

$cat ambari.repo 

#VERSION_NUMBER=2.1.2.1-418
[Updates-ambari-2.1.2.1]
name=ambari-2.1.2.1 - Updates
baseurl=http://public-repo-1.hortonworks.com/ambari/centos6/2.x/updates/2.1.2.1
gpgcheck=1
gpgkey=http://public-repo-1.hortonworks.com/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

question Re: Spark 1.41. installation in Archives of Support Questions (Read Only)

Spark 1.41. installation

Re: Spark 1.41. installation