Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

Spark+R in Cloudera 5.3.0

avatar
Explorer

Please accept my sincere salutations.

 

I have to work with Spark using the R language in Cloudera 5.3.0. I have installed R and also the IDE RStudio in this latter.What will be the next step?What can I do? How should I proceed?

 

I will be grateful if you suggest a tutorial.

 

Best Regards !

1 ACCEPTED SOLUTION

avatar
Contributor

(1) I would start by loading the SparkR package into RStudio so you can make use of it. See the following link under heading "Using SparkR from RStudio" https://github.com/apache/spark/tree/master/R.

 

(2) Now you are ready to run through the following tutorial. However instead of reading the data from "hdfs" load it from your local file system.

http://www.r-bloggers.com/a-first-look-at-spark/ 

 

(3) Study the SparkR Guide to gain more indepth knowledge.

http://spark.apache.org/docs/latest/sparkr.html

 

(4) Study Spark (Dataframes, RDDs, etc) for example with the Oreilly book "Learning Spark". I find that it always helps to understand how something works under the hood. The same holds for SparkR you can easily find some videos about Youtube to understand how it works under the hood, especially the distributed character of SparkR + Spark.

View solution in original post

2 REPLIES 2

avatar
Contributor

(1) I would start by loading the SparkR package into RStudio so you can make use of it. See the following link under heading "Using SparkR from RStudio" https://github.com/apache/spark/tree/master/R.

 

(2) Now you are ready to run through the following tutorial. However instead of reading the data from "hdfs" load it from your local file system.

http://www.r-bloggers.com/a-first-look-at-spark/ 

 

(3) Study the SparkR Guide to gain more indepth knowledge.

http://spark.apache.org/docs/latest/sparkr.html

 

(4) Study Spark (Dataframes, RDDs, etc) for example with the Oreilly book "Learning Spark". I find that it always helps to understand how something works under the hood. The same holds for SparkR you can easily find some videos about Youtube to understand how it works under the hood, especially the distributed character of SparkR + Spark.

avatar
Explorer

Hi,

 

SparkR has been removed from R Cran repository.Could you please advise what needs to be installed instead.

 

Thanks

Sankar