Can someone please help be with the below requirement.
I have 2 topics called feed1 and feed2 in kafka where in feed1 produces the data and feed2 receives the data.
I have written a spark-kafka code wherein consumes data from feed1 and save into HDFS.
Now the requirement is to find out maximum value from the streaming Kafka-spark code and send back the resultant value to the other feed which is feed2.
Ex: feed1 (Produces data)
Spark-Kafka Application --> Consumes data from Feed1 ----> HDFS
Also Find the Max ---> in the same application using 1 min as a window and --------------> Send that max value to the Feed2
Can some one help me whether this can be achieved or not. If yes, please give me some idea or pseudo code.
Thanks in advance