Tutorial: A short primer on Scala - Issue Resolution

[#1]

Tutorial Link

SandboxVersion: HDP 2.3.2
AmbariVersion2.1.2
Hadoop stack version:Hadoop2.7.1.2.3.2.0-2950

Issue 1: Error initializing SparkContext when executing spark-shell command

When you issue the command as root

spark-shell --master yarn-client --driver-memory 512m --executor-memory 512m

You would would receive the error:

ERROR SparkContext: Error initializing SparkContext.org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root/.sparkStaging/application_1451921066894_0001":hdfs:hdfs:drwxr-xr-x

Resolution

sudo su - hdfs
hdfs dfs -mkdir /user/root
hdfs dfs -chown root:hdfs /user/root
exit

--------------------------------------------------------

Issue 2

[#2]

@Ancil McBarnett Whats the issue 2?

[#3]

Please note, this tutorial is deprecated with HDP 2.4 release.