Support Questions

Find answers, ask questions, and share your expertise

unable to load spark shell

avatar
Explorer
spark-shell
File "/bin/hdp-select", line 256
print "ERROR: Invalid package - " + name
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("ERROR: Invalid package - " + name)?
ls: cannot access /usr/hdp//hadoop/lib: No such file or directory
Exception in thread "main" java.lang.IllegalStateException: hdp.version is not set while running Spark under HDP, please set through HDP_VERSION in spark-env.sh or add a java-opts file in conf with -Dhdp.version=xxx
at org.apache.spark.launcher.Main.main(Main.java:118)
1 ACCEPTED SOLUTION

avatar
Explorer

Set spark home and Python version 2.7 in bash.rc

 

 

View solution in original post

3 REPLIES 3

avatar
Expert Contributor

hi @Sadique1 

 

Can you run which spark-shell and share us the results

avatar
Explorer

[root@datascience38 ~]# which spark-shell

/opt/anaconda2/bin/spark-shell

avatar
Explorer

Set spark home and Python version 2.7 in bash.rc