- Subscribe to RSS Feed
- Mark Question as New
- Mark Question as Read
- Float this Question for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page
Attempting to write remote Spark Streaming job to CDH 5.1.3 HDFS
Created on ‎10-29-2014 09:34 PM - edited ‎09-16-2022 02:11 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have been trying to find some Spark Streaming class or config to enable me to present permission credentials or something of that nature in order to use .SaveAsTextFile(HDFSPath) to write to CDH HDFS. Is there some way to assign my Spark Streaming super user permissions? Thanks for any advice!
Created ‎10-30-2014 04:38 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The spark app will run as whatever user you submitted it as, or should be. I would just make the directory writable to that user if at all possible.
Created ‎10-30-2014 04:38 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The spark app will run as whatever user you submitted it as, or should be. I would just make the directory writable to that user if at all possible.
