question Re: How do I tell which version ofSpark I am running? in Archives of Support Questions (Read Only)

How do I tell which version ofSpark I am running?

ed_day — Mon, 05 Sep 2016 22:13:24 GMT

I have finally got HDP 2.3 to run! Now I am trying to install zeppelin via these instructions:

http://hortonworks.com/hadoop-tutorial/apache-zeppelin/

but I need to know which version of Spark I am running. How do I find this in HDP?

TIA!

Re: How do I tell which version ofSpark I am running?

pankaj_singh — Mon, 19 Aug 2019 08:50:06 GMT

@ed day You can get the version information from Ambari UI. Click on Admin -> Stack and Versions and you will find the version information under Version tab.

Re: How do I tell which version ofSpark I am running?

anandi — Mon, 05 Sep 2016 23:16:50 GMT

The most easy way just launch "spark-shell" at the command line. This will give you the active version running on your cluster:

[root@xxxxxxx ~]# spark-shell 16/09/05 17:15:15 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 1.3.1 /_/

Using Scala version 2.10.4 (OpenJDK 64-Bit Server VM, Java 1.7.0_71) Type in expressions to have them evaluated. Type :help for more information.

[root@xxxxxxx ~]# spark-shell
16/09/05 17:15:15 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.3.1
      /_/

Using Scala version 2.10.4 (OpenJDK 64-Bit Server VM, Java 1.7.0_71)
Type in expressions to have them evaluated.
Type :help for more information.

Re: How do I tell which version ofSpark I am running?

musushakeel — Thu, 14 Mar 2019 23:28:23 GMT

sc.version()

spark -submit --version