How to : HTTP Stream in flink

[#1]

Which methods/objects does Flink provide in order to enable users to read HTTP streams?

[#2]

@Hemant Kumar

The best information on this topic can be found on flink site. https://flink.apache.org/news/2015/02/09/streaming...

https://github.com/dataartisans/flink-streaming-de...

Not sure if it helps to customize for HTTP

[#3]

Thanks @Neeraj Sabharwal

For HTTP, I am in a state where I have to write a custom SourceFunction. Before I try it, I am exploring if there is a possibility to Handle HTTP Requests by using any of the existing StreamExecutionEnvironment functions/objects.

I am also trying to handle HTTP Stream for Flink using NiFi Connector as mentioned here

I hope either of these will help me in achieving desired results. 🙂

[#4]

+ @Robert Metzger

[#5]

Sorry for the late reply. As I said in the other thread, I would recommend using a SourceFunction here. Feel free to post your code here as well, I can review it and give some hints for improving it.

[#6]

Thank you for posting the code here @Hemant Kumar. Sorry for the delay, I've now looked at and it looks good. If you want to optimize the code, I would try to move the code from the FlinkJSONObject constructor into the HTTPJSONStream class. It seems that the do/while loop is actually controlled from within the FlinkJSONObject.

[#7]

Thanks a lot @Robert Metzger