Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope) at [Source: {"id":"8","name":"print"}; line: 1, column: 1]

Highlighted

com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope) at [Source: {"id":"8","name":"print"}; line: 1, column: 1]

New Contributor

I am trying to run a sample spark streaming application where kafka is the streaming source in Zeppline.

I am following blog link

code:-

%spark
import _root_.kafka.serializer.DefaultDecoder
import _root_.kafka.serializer.StringDecoder
import org.apache.spark.streaming.kafka.KafkaUtils
import org.apache.spark.storage.StorageLevel
import org.apache.spark.streaming._
import org.apache.spark.streaming.{Seconds, StreamingContext}
import org.apache.spark.rdd._
// prevent INFO logging from pollution output
sc.setLogLevel("ERROR")
// creating the StreamingContext with 5 seconds interval
val ssc = new StreamingContext(sc, Seconds(5))
val kafkaConf = Map(
    "metadata.broker.list" ->"xx.xx.xx.xx:6667,xx.xx.xx.xx:6667" ,
    "zookeeper.connect" -> "xx.xx.xx.xx:2181" ,
    "group.id" -> "fbczepplinconsumer",
    "zookeeper.connection.timeout.ms" -> "1000"
)
val lines = KafkaUtils.createStream[Array[Byte], String, DefaultDecoder, StringDecoder](
    ssc,
    kafkaConf,
    Map("fbc-dev" -> 1),   // subscripe to topic and partition 1
    StorageLevel.MEMORY_ONLY
)
val words = lines.flatMap{ case(x, y) => y.split(" ")}
words.print()
ssc.start()
ssc.awaitTermination(5000)

I am getting following error

com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
 at [Source: {"id":"2","name":"print"}; line: 1, column: 1]

Don't have an account?
Coming from Hortonworks? Activate your account here