Support Questions

Find answers, ask questions, and share your expertise
Announcements
Celebrating as our community reaches 100,000 members! Thank you!

spark streaming json to hive

avatar
Expert Contributor

Hi All,

I am beginner to spark and wanted to do the below.

a port 55500 is trying to send jsons as a stream (ex: {"one":"1","two":"2"}{"three":"3","four":"4"}).

I have a orc table in hive with columns given below

one, two,three,four,spark_streaming_startingtime,spark_streaming_endingtime,partition_value

I want to load the streaming values in to hive orc table.

Can you please guide me how to achieve this.

Thank you for your support.

1 ACCEPTED SOLUTION

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login
4 REPLIES 4

avatar
hide-solution

This problem has been solved!

Want to get a detailed solution you have to login/registered on the community

Register/Login

avatar
Expert Contributor
@Felix Albani

Thank your for quick response, I will go through the given info

avatar
Expert Contributor
@Felix Albani

Can you help me with the pyspark version of the above please.

avatar

@Mark sure, here is the link to the pyspark network word count example:

https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/network_wordcount.py

HTH