question Differnt Spark versions on HDP 2.4.0.0-169 in Archives of Support Questions (Read Only)

Differnt Spark versions on HDP 2.4.0.0-169

Int17 — Thu, 11 Aug 2016 17:03:14 GMT

Hello,

in this HDP version Spark 1.6.0.2.4 was installed during cluster installation. Now we want to play with SAP VORA 1.2 bad sadly will work with Spark 1.5.1 or 1.5.2 only. I believe I have 2 options:

1) Deinstall Spark 1.6 and install the 1.5.2 version. Problem: from where can I download a package ready to use for Amabri

2) Installing it manually in a different directory (e.g. /usr/apache/spark-1.5.2) from http://d3kbcqa49mib13.cloudfront.net/spark-1.5.2-bin-hadoop2.4.tgz. Problem: How to setup a 1.5.2 environment with no effects on the installed 1.6 version.

🙂 Klaus

Re: Differnt Spark versions on HDP 2.4.0.0-169

aervits — Sun, 14 Aug 2016 00:21:46 GMT

You can't install Spark 1.5.2 on HDP 2.4.x unless you build it from source and deploy it yourself. you can get the sides from Apache Spark webpage. as long as you keep your Spark init scripts in a non-hdp specific directory you should be good. make sure /usr/bin/spark does not collide with HDP spark. It's best if you wait for SAP Bods 1.3 which I believe will support spark 1.6. in the next major HDP release will be able to run two independent versions of spark simultaneously (1.6.2 and 2.0 technical preview).

Re: Differnt Spark versions on HDP 2.4.0.0-169

Int17 — Sun, 14 Aug 2016 19:05:42 GMT

Hello Artem,

many thanks for your answer.

🙂 Klaus