Archives of Support Questions (Read Only)

This is an archived board for historical reference. Information and links may no longer be available or relevant
Announcements
This board is archived and read-only for historical reference. To ask a new question, please post a new topic on the appropriate active board.

Attempting to write remote Spark Streaming job to CDH 5.1.3 HDFS

avatar
Explorer

I have been trying to find some Spark Streaming class or config to enable me to present permission credentials or something of that nature in order to use .SaveAsTextFile(HDFSPath) to write to CDH HDFS.  Is there some way to assign my Spark Streaming super user permissions?  Thanks for any advice!

1 ACCEPTED SOLUTION

avatar
Master Collaborator

The spark app will run as whatever user you submitted it as, or should be. I would just make the directory writable to that user if at all possible.

View solution in original post

1 REPLY 1

avatar
Master Collaborator

The spark app will run as whatever user you submitted it as, or should be. I would just make the directory writable to that user if at all possible.