Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

How to use spark older version

SOLVED Go to solution
Highlighted

How to use spark older version

Contributor

Hi Team,


We upgraded our spark to 2.3.0.2.6.5.0-292. When I export the spark_major_version=2, and if I type spark-shell then it will be connecting to 2.3.0.2.6.5.0-292. I want to use spark 2.1.0.2.6.0.23-2 version, how can I do that. Please help us on this.

1 ACCEPTED SOLUTION

Accepted Solutions

Re: How to use spark older version

Contributor

Hi Team,


Got the answer, we have to use SPARK_HOME and then run the spark-shell.

export SPARK_HOME=/usr/hdp/2.6.0.23-2/spark2

export SPARK_MAJOR_VERSION=2

spark-shell

then it will connect to 2.6.0.23


1 REPLY 1

Re: How to use spark older version

Contributor

Hi Team,


Got the answer, we have to use SPARK_HOME and then run the spark-shell.

export SPARK_HOME=/usr/hdp/2.6.0.23-2/spark2

export SPARK_MAJOR_VERSION=2

spark-shell

then it will connect to 2.6.0.23