Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

streaming transaction data from oracle

streaming transaction data from oracle

Contributor

Hi All,

 

       Is there any way to stream data from oracle table to hdfs in near real time.

1 REPLY 1
Highlighted

Re: streaming transaction data from oracle

Explorer
I'm new to Hadoop so I use StreamSets to transfer data from Oracle to hdfs. This is assuming there are PKs and Timestamps for the offsets. Caveat is if the buffer is not complete, your records are not real-time until that happens.

The other option is to write a Spark job, include JDBC driver, and write the file to HDFS, see this for a post written in 2015 that can be rewritten https://www.rittmanmead.com/blog/2015/08/combining-spark-streaming-and-data-frames-for-near-real-tim...