Trying to connect to Spark Cluster from RStudio on local machine.
I go to set up the connection. From the drop down menu for "Master:", I choose "Cluster'' and then I receive this message attached.
Connecting with a remote spark cluster requires an RStudio server instance...
I am very new to this. Can anyone shed any light on how to overcome this problem?
I havn't used R studio.
But if you are looking for a notebook to launch spark jobs, then you can give a try to Apache Zeppelin.
HDP comes bundled with zeppelin and you can install it as a service on the cluster.
Please use https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.0/bk_zeppelin-component-guide/content/ch_inst... to install zeppelin using ambari.
Once you have zeppelin, you can use 'R interpreter' and start interacting with spark.
Steps to configure R interpreter : https://zeppelin.apache.org/docs/0.6.2/interpreter/r.html