question unable to load spark shell in Support Questions

unable to load spark shell

Sadique1 — Tue, 05 Oct 2021 09:21:29 GMT

spark-shell
File "/bin/hdp-select", line 256
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
ls: cannot access /usr/hdp//hadoop/lib: No such file or directory
Exception in thread "main" java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx
at org.apache.spark.launcher.Main.main(Main.java:118)

Re: unable to load spark shell

Shifu — Tue, 05 Oct 2021 13:07:43 GMT

hi @Sadique1

Can you run which spark-shell and share us the results

Re: unable to load spark shell

Sadique1 — Wed, 06 Oct 2021 06:40:15 GMT

[root@datascience38 ~]# which spark-shell

/opt/anaconda2/bin/spark-shell

Re: unable to load spark shell

Sadique1 — Mon, 11 Oct 2021 08:50:13 GMT

Set spark home and Python version 2.7 in bash.rc