Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

spark streaming socket read issue

spark streaming socket read issue

New Contributor

hi there, I have a spark streaming issue that i am not able to figure out , below code reads from a socket, but I don't see any input going into the job, I have nc -l 1111 running, and dumping data though, not sure why my spark job is not able to read data from 10.176.110.112:1111.Please advice.

Dataset<Row> d = sparkSession.readStream().format("socket") .option("host", "10.176.110.112").option("port", 1111).load();

thanks

Pradeep

1 REPLY 1

Re: spark streaming socket read issue

Rising Star

You should use:

nc -lk 1111

to send data, in order to be able to read it with Spark Streaming. Mind the k.

Don't have an account?
Coming from Hortonworks? Activate your account here