Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: The Cloudera Community will undergo maintenance on Saturday, August 17 at 12:00am PDT. See more info here.

Simple Scala code not working in Spark-shell repl...

Simple Scala code not working in Spark-shell repl...

New Contributor

I have CDH 5.1 installed on a 5 node cluster.  I am building up a spark program from a series of REPL commands but am experiencing unexpected behaviour 

the commnads are as follows 

 

case class Reading(accountId: String, senderId: String, sensorId: String, metricName: String , readTime: Long , value: Double)

var r2 =Reading("sss","fff","FGGF","hjjj", 232L, 22.3)

import scala.collection.mutable.ListBuffer

var readings = ListBuffer[Reading]()

readings.append(r2)

 

The last line thows a mismatch error

 

<console>:18: error: type mismatch;
found : Reading
required: Reading
readings.append(r2)

 

This works as expected from a standalone instance of spark and scala  on an Ubuntu box , CDH is installed on Centos  where java versions differ  , Centos uses oracle jdk, whilst Ubuntu uses OpenJDK   .   I have tried the code on several instances of CDH that we have here locally and the issue is the same, 

 

I would expect the spark-shell repl to have full scala functionality ,  would this be  a valid assumption ?  

 

If some else could try the same commands on a CDH instance I would be grateful  to know if it worked as expected or not

 

vr

Hugh McBride

1 REPLY 1

Re: Simple Scala code not working in Spark-shell repl...

Master Collaborator
Using scala version 2.11.8, no error is thrown.

scala> readings
res1: scala.collection.mutable.ListBuffer[Reading] = ListBuffer(Reading(sss,fff,FGGF,hjjj,232,22.3))