question Re: I need to create a spark streaming application which should monitor a hdfs path continuously and read only the new incoming csv data in csv tabular format. in Support Questions https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215563#M177473 <P>Thanks <A rel="user" href="https://community.cloudera.com/users/11048/falbani.html" nodeid="11048">@Felix Albani</A>. I found it. For Scala code pls check this git hub link <A href="https://github.com/phatak-dev/spark2.0-examples/blob/master/src/main/scala/com/madhukaraphatak/examples/sparktwo/streaming/FileStreamExample.scala" target="_blank">https://github.com/phatak-dev/spark2.0-examples/blob/master/src/main/scala/com/madhukaraphatak/examples/sparktwo/streaming/FileStreamExample.scala</A>. This is exactly what i needed. </P> Thu, 09 Aug 2018 18:20:47 GMT sganeshkumar 2018-08-09T18:20:47Z I need to create a spark streaming application which should monitor a hdfs path continuously and read only the new incoming csv data in csv tabular format. https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215561#M177471 Tue, 07 Aug 2018 16:02:45 GMT https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215561#M177471 sganeshkumar 2018-08-07T16:02:45Z Re: I need to create a spark streaming application which should monitor a hdfs path continuously and read only the new incoming csv data in csv tabular format. https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215562#M177472 <P> <A rel="user" href="https://community.cloudera.com/users/89308/sganeshkumar.html" nodeid="89308">@Sudharsan Ganeshkumar </A></P><P>Out of the box spark provides the fileStream. You can read more here:</P><P><A href="https://spark.apache.org/docs/latest/streaming-programming-guide.html" target="_blank">https://spark.apache.org/docs/latest/streaming-programming-guide.html</A></P><P>HTH</P><P>*** If you found this answer addressed your question, please take a moment to login and click the "accept" link on the answer.</P> Tue, 07 Aug 2018 20:13:56 GMT https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215562#M177472 falbani 2018-08-07T20:13:56Z Re: I need to create a spark streaming application which should monitor a hdfs path continuously and read only the new incoming csv data in csv tabular format. https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215563#M177473 <P>Thanks <A rel="user" href="https://community.cloudera.com/users/11048/falbani.html" nodeid="11048">@Felix Albani</A>. I found it. For Scala code pls check this git hub link <A href="https://github.com/phatak-dev/spark2.0-examples/blob/master/src/main/scala/com/madhukaraphatak/examples/sparktwo/streaming/FileStreamExample.scala" target="_blank">https://github.com/phatak-dev/spark2.0-examples/blob/master/src/main/scala/com/madhukaraphatak/examples/sparktwo/streaming/FileStreamExample.scala</A>. This is exactly what i needed. </P> Thu, 09 Aug 2018 18:20:47 GMT https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215563#M177473 sganeshkumar 2018-08-09T18:20:47Z Re: I need to create a spark streaming application which should monitor a hdfs path continuously and read only the new incoming csv data in csv tabular format. https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215564#M177474 <P><A rel="user" href="https://community.cloudera.com/users/89308/sganeshkumar.html" nodeid="89308">@Sudharsan Ganeshkumar</A> if my answer has helped you please remember to login and mark it as accepted. </P> Fri, 10 Aug 2018 00:41:58 GMT https://community.cloudera.com/t5/Support-Questions/I-need-to-create-a-spark-streaming-application-which-should/m-p/215564#M177474 falbani 2018-08-10T00:41:58Z