question Error while creating a DataStream using fromElements function in Archives of Support Questions (Read Only)

Error while creating a DataStream using fromElements function

hemant_dindi — Fri, 16 Sep 2022 09:54:06 GMT

Below is the expeption -

Caused by: java.io.IOException: Failed to deserialize an element from the source. If you are using user-defined serialization (Value and Writable types), check the serialization functions. Serializer is org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer@599fcdda at org.apache.flink.streaming.api.functions.source.FromElementsFunction.run(FromElementsFunction.java:121) at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58) at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:55) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:218) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:584) at java.lang.Thread.run(Thread.java:745)

Re: Error while creating a DataStream using fromElements function

jstraub — Mon, 21 Dec 2015 14:04:00 GMT

@Ali Bajwa have you seen sth. like this before?

Re: Error while creating a DataStream using fromElements function

abajwa — Mon, 21 Dec 2015 14:16:36 GMT

+ @Robert Metzger @Stephan Ewen as they are the experts on Flink streaming

@Hemant Kumar this also may provide some hints: http://stackoverflow.com/questions/34379170/error-...

Re: Error while creating a DataStream using fromElements function

hemant_dindi — Mon, 21 Dec 2015 16:11:04 GMT

Thanks @Ali Bajwa

@Robert Metzger @Stephan Ewen I was trying to read a Web log which is accessible through a HTTP URL. I used a InputStreamReader to read the content from the HttpConnection. As InputStreamReader is not serializable, I got the above exception.

Should we prepare a custom SourceFunction to handle HTTP Streams or does Flink provide any methods/objects in order to enable users to read HTTP streams?

Re: Error while creating a DataStream using fromElements function

rmetzger1 — Mon, 21 Dec 2015 23:30:21 GMT

Right now, there are not methods for reading HTTP streams. But implementing a custom "SourceFunction" (as you suggested) is the recommended appropach. Please let me know if you have further questions.

Re: Error while creating a DataStream using fromElements function

aervits — Tue, 02 Feb 2016 22:58:01 GMT

@Hemant Kumar has this been resolved? Can you post your solution or accept best answer?