Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

can we write MapReduce job to read data from oracle and MySql?

can we write MapReduce job to read data from oracle and MySql?

Expert Contributor

I want to read data from Oracle and Mysql. Then I need to Compare data which are read from Oracle and Mysql and the output will be stored in MSSQL server. I want to perform all this using MapReduce job in c#.

If I have configured Hadoop multinode cluster and I am executing MapReduce job to read data from Oracle and MySQL, data which MapReduce reads will distribute across the memory of multiple nodes in the cluster.?

1 REPLY 1
Highlighted

Re: can we write MapReduce job to read data from oracle and MySql?

Yess you can use MR to do all you have said OR you can use some inbuilt tools .

  1. Use Sqoop to read data from DB and dump it on to HDFS.
  2. Use MR to read data from HDFS and perform the business logic.
  3. The output of MR is saved in HDFS and again using SQoop pushed into DB.

Sqoop is a wrapper over MapReduce to pull/push data to Database. You can always write your own custom MapReduce to do the same.


Don't have an account?
Coming from Hortonworks? Activate your account here