Created 05-11-2020 06:00 AM
Hello,
I want to send messages from kafka to Spark and then use Spack SQL for from manupulation. Finally i want to send it to another Kafka topic.
When i use
kvs = KafkaUtils.createStream(ssc,"localhost:2181", "spark-streaming-consumer", {topic:1})
df = spark.readStream.format("kafka").option("kafka.bootstrap.servers","localhost:9092").option("subscribe", "SparkPublish").load()
Even after this i am getting different errors.
Can anyone tell me how can receive a message from kafka using Spark streaming as Dataframes and use SparkSQl on it?
Thank You.