Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Setup HDP 2.4 from scratch with SparkR, requirements

Highlighted

Setup HDP 2.4 from scratch with SparkR, requirements

Explorer

Hi, If I install HDP 2.4.x from scratch, and I want to use SparkR, do I need to install also an R distribution on the workers node?

Without R I'm able to use spark-shell from a node outside the cluster and everithing seems good (run Pi Estimation example without problem).

I am also able to use sparklyr library and do stuff, but if I understood well, it's because the sparklyr library perform some kind of translation of R code in something like scala.

When I tried to use SparkR library, it seems like it search the Rscript executable on the cluster. Do I need to install it on the workers or master node of the cluster? Or it's a problem with the cluster configuration? Or is a problem with the "submitter" outside of the cluster?

Bests,

Sergio L.

2 REPLIES 2
Highlighted

Re: Setup HDP 2.4 from scratch with SparkR, requirements

Expert Contributor
Highlighted

Re: Setup HDP 2.4 from scratch with SparkR, requirements

Expert Contributor
Don't have an account?
Coming from Hortonworks? Activate your account here