Support Questions

Find answers, ask questions, and share your expertise

Attempting to write remote Spark Streaming job to CDH 5.1.3 HDFS

avatar
Explorer

I have been trying to find some Spark Streaming class or config to enable me to present permission credentials or something of that nature in order to use .SaveAsTextFile(HDFSPath) to write to CDH HDFS.  Is there some way to assign my Spark Streaming super user permissions?  Thanks for any advice!

1 ACCEPTED SOLUTION

avatar
Master Collaborator

The spark app will run as whatever user you submitted it as, or should be. I would just make the directory writable to that user if at all possible.

View solution in original post

1 REPLY 1

avatar
Master Collaborator

The spark app will run as whatever user you submitted it as, or should be. I would just make the directory writable to that user if at all possible.