Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

5.1 Quickstart VM issue

5.1 Quickstart VM issue

New Contributor

Assuming spark-shell should be run as the default cloudera user, I was getting:

 

...

14/08/09 17:44:31 INFO HttpServer: Starting HTTP Server
14/08/09 17:44:31 INFO SparkUI: Started SparkUI at http://192.168.122.77:4040
14/08/09 17:44:32 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
org.apache.hadoop.security.AccessControlException: Permission denied: user=cloudera, access=EXECUTE, inode="/user/spark":spark:spark:drwxr-x---
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271)

...

 

Then all kinds of errors.

 

I ran this to fix it:

$ sudo su - hdfs

$ hadoop fs -chmod -R 777 /user/spark

8 REPLIES 8
Highlighted

Re: 5.1 Quickstart VM issue

Master Collaborator

I don't see this when I start the 5.1  Quickstart VM. I used the VirtualBox image. The user is logged in as 'cloudera' and you run spark-shell as 'cloudera', but it does not need to write anything to '/user/spark'. Is it possible it was a problem with modifications you made, or was it out of the box? 

Highlighted

Re: 5.1 Quickstart VM issue

New Contributor
This is right out of the box with the KVM image.

Nothing at all was changed.

I don't have enough RAM to try it on my Windows machine.

Marco

Highlighted

Re: 5.1 Quickstart VM issue

Master Collaborator

Hm, the only other thing I did when I started it was to log in to Hue. I doubt that has any effect. I assume you're also using the latest VM? Does /user/cloudera exist in HDFS for you at startup?

Highlighted

Re: 5.1 Quickstart VM issue

New Contributor
I downloaded the latest VM yesterday.

In hdfs:/user, I see a cloudera (755), history (755), hive (777), hue
(775), oozie (755) and spark (777 now) directory. All user and group owners
match the directory names.

I'm not concerned about this now, basic Spark commands work, so I'm good.

Highlighted

Re: 5.1 Quickstart VM issue

New Contributor

Same issue for me with Cloudera VM 5.1.0-1 on VirtualBox, Marco's chmod works around it.

Highlighted

Re: 5.1 Quickstart VM issue

New Contributor

Still an issue for for Cloudera VM 5.2.0 on VirtualBox.  Marco's chmod solution worked a treat, thanks.

Highlighted

Re: 5.1 Quickstart VM issue

New Contributor

Can you please tell me what the Macro's chmod is? I just started working with vm and java. I'd really appreciate your help.

Highlighted

Re: 5.1 Quickstart VM issue

New Contributor

Hi mankad,

My reference is to the post at the top of this thread by marco.shaw where he details running the following commands that sorted things out for me:

$ sudo su - hdfs
$ hadoop fs -chmod -R 777 /user/spark

Hope that helps.

Don't have an account?
Coming from Hortonworks? Activate your account here