I am trying to use SparkR on Zeppelin (running the HDP 2.6 Sandbox) but it is not working, and also not returning any kind of error message. For example, using the included example notebook "R (SparkR)" and running the first simple command does nothing. It runs for a few seconds and then sets the section status to "ERROR", but does not return an error message.
%spark2.r foo <- TRUE print(foo) bare <- c(1, 2.5, 4) print(bare) double <- 15.0 print(double)
Surely if this is included as an example Notebook the necessary configurations should be in place by default? Or is there something I am missing that still needs to be set?
I have no problems running Spark commands. Only SparkR is giving me trouble.
Many thanks in advance.
Can you check:
- you are able to run a sparkR shell, e.g. /usr/hdp/current/spark2-client/bin/sparkR
- in the /var/log/spark2, /var/log/spark, /var/log/zeppelin directories, are there any error messages.
- all the related services and components are running (Spark, Spark2, Zeppelin, HDFS, etc.)
- what R packages are installed?
@Karl, have you tried running SparkR directly, without Zeppelin? That might help identify whether it's a Zeppelin or Spark(R) issue. Here's documentation for running SparkR with R from the command line: