Reply
Highlighted
Expert Contributor
Posts: 70
Registered: ‎11-24-2017

sparklyr and SparkR support on Oozie

Hi everyone!

 

Basically I have this R script (written from some one else..) which uses some local R libraries to perform clustering on data coming form Hive tables. I need to schedule it on the cluster using Oozie; my idea was to launch it on a spark action through a sparkR or sparklyr script.

 

Does anyone have any experience on scheduling Oozie workflows with spark actions using sparklyr or SparkR scripts? Does it work?

 

I have used Spark with scala and python but never with R.

Thanks in advance for sharing any information!

 

 

Announcements