Support Questions

Find answers, ask questions, and share your expertise
Celebrating as our community reaches 100,000 members! Thank you!

Run SparkR | or R package on my Cloudera 5.9 Spark

Expert Contributor



I have 3 node cluster having Cloudera 5.9 running on CentOS 6.7. I need to connect my R packages (running on my Laptop) to the Spark runing in cluster mode on Hadoop.


However If I try to connect the local R through Sparklyr Connect to Hadoop Spark it is giving Error. As it is searching the Spark home on the laptop itself.


I googled and found we can install SparkR and use R with Spark. However I have few questions regarding the same.


  1. I have downloaded the tar file from But my question is I directly copy it to my Linux server and install?
  2. Do I have to Stop/delete my existing Spark which is NOT Stand Alone and using Yarn i.e. it is running in Cluster mode? or SparkR can just run on top of it, If I install it on the server?
  3. Or do I have to run Spark on Stand Alone mode (get Spark gateways running and Start master/slave using script) and install the package from linux command line on top of it?
  4. If it get installed will I be able to access it through CM UI?

Please help, I am new in this and really need guidance.





Expert Contributor

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community


New Contributor

Hi AutoIN, 


This link is opening fine
but Step f under installation : link is not working as expected

Can you please provide the location we are using CDH 6.3.3 ans spark version is 2.4.0


Master Collaborator

Hello @PR_224 


Please replace steps f to j with what @singh101 suggested in one of the above comments: . The idea is - we make use of the binaries from the CDH parcel, instead of downloading it from upstream.


On a side note: CDP Base provides sparkR out of the box (in case if you plan to upgrade in near future)


Good luck!

New Contributor

Thank you AutoIN it worked before only.  Took more time to respond. 

Master Collaborator

No worries @PR_224 
Glad it's fixed : )