Reply
Explorer
Posts: 15
Registered: ‎08-11-2014

/user/spark/spark2ApplicationHistory folder permissions is wrong during spark2.2 installation

Hi

We're using CDH 5.10.1 on RHEL7.

we've started with default the spark 1.6

and then we've decided to go with spark 2.2

 

When we create a cluster CDH 5.10.1 with default spark and then we upgrade to spark 2.2 (add percel, add csd, and keep following innstructions), the folder /user/spark/spark2ApplicationHistory has permissions "drwxrwxrwt"

 

However, whne we install the cluster and install directly spark 2.2, without the default one first, the same folder gets "drwxr-xr-x" permissions. (both time owner and group are "spark")

i guess some chmod command is missing during the installation path of spark2.2, or do we do something incorrect?

 

did someone got same issues?

 

this happened in our testing environments and caused some spark jobs to fail, until we changed the permissions mode of the folder.

 

thanks

Doron

Announcements