Community Articles
Find and share helpful community-sourced technical articles
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.
Labels (1)

Tutorial Link

SandboxVersion: HDP 2.3.2
AmbariVersion2.1.2
Hadoop stack version:Hadoop2.7.1.2.3.2.0-2950

Issue 1: Error initializing SparkContext when executing spark-shell command

When you issue the command as root

spark-shell --master yarn-client --driver-memory 512m --executor-memory 512m

You would would receive the error:

ERROR SparkContext: Error initializing SparkContext.org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="/user/root/.sparkStaging/application_1451921066894_0001":hdfs:hdfs:drwxr-xr-x

Resolution

sudo su - hdfs
hdfs dfs -mkdir /user/root
hdfs dfs -chown root:hdfs /user/root
exit

--------------------------------------------------------

Issue 2

1,208 Views
Comments

@Ancil McBarnett Whats the issue 2?

Please note, this tutorial is deprecated with HDP 2.4 release.

Don't have an account?
Coming from Hortonworks? Activate your account here
Version history
Revision #:
1 of 1
Last update:
‎01-04-2016 04:55 PM
Updated by:
 
Contributors
Top Kudoed Authors