Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

how to create spark output file with owner full permissions

Highlighted

how to create spark output file with owner full permissions

I am creating spark output files using sqlcontext but the files created have only read permissions , how would I create file with full permissions . I tried the below but no luck, please help. I am using spark 1.4.1 with scala.

val sc = new SparkContext(sparkConf)

val hConf = sc.hadoopConfiguration

hConf.set("fs.s3.canned.acl","BucketOwnerFullControl")

1 REPLY 1
Highlighted

Re: how to create spark output file with owner full permissions

Contributor

In the job, you have to access the JavaSparkContext and from there get the Hadoop configuration and set the parameter there. For example:

sc._jsc.hadoopConfiguration().set('fs.s3.canned.acl','BucketOwnerFullControl')
Don't have an account?
Coming from Hortonworks? Activate your account here