Spark 2 in zeppelin cannot access local files due to permissions

[#1]

Hi,

I'm using spark2 in Zeppelin. (HDP 2.6.4)
I'm loading a csv file with spark.read

val test = spark.read      .option("header", "false")  .option("delimiter", ",")  .option("inferSchema", "true")  .csv("file:///root/share/test.csv")

/root/share/ is a shared folder from host to docker container (virtualbox).
it has privileges like dwxrwxr---

Because of the shared folder, I cannot chmod 777 on it (or maybe someone knows how to do this...)

Anyways, spark comes back with a permission denied on this forlder/file.

Questions:

1. when running such a spark2 request in zeppelin, which permissions are applied?
2. How can I "elevate" these permissions to root (so that access can be made to any folder)

Thnaks a lot for your help!

[#2 · Accepted solution]

@Arnaud Bohelay

Answers inline:

1. when running such a spark2 request in zeppelin, which permissions are applied?

Answer > The application will run as zeppelin user, so all access will be performed as zeppelin

2. How can I "elevate" these permissions to root (so that access can be made to any folder)

Answer > This is generally not a good idea. I suggest you create a folder with proper access permissions somewhere else. Maybe under opt. Like this:

mkdir /opt/share

chmod 777 /opt/share

and use that one instead of /root which is the home directory for the root user.

HTH

*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.

[#3]

@Arnaud Bohelay Please let me know the above has helped answer your question. Thanks.