Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Quickstart tutorial exercise 3 spark-shell permission denied error

Highlighted

Quickstart tutorial exercise 3 spark-shell permission denied error

New Contributor

Working through the tutorial, got this error when i ran the spark-shell comming in the tutorial:

spark-shell --master yarn-client

 

16/08/15 17:40:10 ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=WRITE, inode="/user/spark/applicationHistory":spark:supergroup:drwxr-xr-x

stack trace ....

Ending with these:

<console>:16: error: not found: value sqlContext
import sqlContext.implicits._
<console>:16: error: not found: value sqlContext
import sqlContext.sql

scala>

 

Fixed it by giving the cloudera access to the hadoop filesystem folder with this command:

sudo -u hdfs hadoop fs -chmod 777 /user/spark/applicationHistory

 

Then the rest of the exercise worked.

 

Joe

 

Don't have an account?
Coming from Hortonworks? Activate your account here