Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark 2 in zeppelin cannot access local files due to permissions

avatar
Explorer

Hi,

I'm using spark2 in Zeppelin. (HDP 2.6.4)
I'm loading a csv file with spark.read

val test = spark.read      .option("header", "false")  .option("delimiter", ",")  .option("inferSchema", "true")  .csv("file:///root/share/test.csv")

/root/share/ is a shared folder from host to docker container (virtualbox).
it has privileges like dwxrwxr---

Because of the shared folder, I cannot chmod 777 on it (or maybe someone knows how to do this...)

Anyways, spark comes back with a permission denied on this forlder/file.

Questions:

1. when running such a spark2 request in zeppelin, which permissions are applied?
2. How can I "elevate" these permissions to root (so that access can be made to any folder)

Thnaks a lot for your help!

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
2 REPLIES 2

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar

@Arnaud Bohelay Please let me know the above has helped answer your question. Thanks.