question Re: Spark+R in Cloudera 5.3.0 in Archives of Support Questions (Read Only)

Spark+R in Cloudera 5.3.0

RMG — Fri, 16 Sep 2022 10:05:08 GMT

Please accept my sincere salutations.

I have to work with Spark using the R language in Cloudera 5.3.0. I have installed R and also the IDE RStudio in this latter.What will be the next step?What can I do? How should I proceed?

I will be grateful if you suggest a tutorial.

Best Regards !

Re: Spark+R in Cloudera 5.3.0

MVERVUURT — Wed, 06 Jul 2016 09:43:18 GMT

(1) I would start by loading the SparkR package into RStudio so you can make use of it. See the following link under heading "Using SparkR from RStudio" https://github.com/apache/spark/tree/master/R.

(2) Now you are ready to run through the following tutorial. However instead of reading the data from "hdfs" load it from your local file system.

http://www.r-bloggers.com/a-first-look-at-spark/

(3) Study the SparkR Guide to gain more indepth knowledge.

http://spark.apache.org/docs/latest/sparkr.html

(4) Study Spark (Dataframes, RDDs, etc) for example with the Oreilly book "Learning Spark". I find that it always helps to understand how something works under the hood. The same holds for SparkR you can easily find some videos about Youtube to understand how it works under the hood, especially the distributed character of SparkR + Spark.

Re: Spark+R in Cloudera 5.3.0

Sankar — Tue, 29 Jan 2019 14:31:11 GMT

Hi,

SparkR has been removed from R Cran repository.Could you please advise what needs to be installed instead.

Thanks

Sankar