Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

historicize Oracle RDBMS with Hadoop

Highlighted

historicize Oracle RDBMS with Hadoop

Hi, in a scenario where you need to purge data from an Oracle RDBMS ( delete data from online DATABASE and upload them on the historical DATABASE ) , you can use Hadoop ?

​Thanks in advance

 

Michele

6 REPLIES 6
Highlighted

Re: historicize Oracle RDBMS with Hadoop

Master Collaborator

Apache Sqoop can be used to move data from Oracle into HDFS, where batch processing of various kinds can be done, and then from HDFS back into an Oracle database. Is that what you have in mind?

Highlighted

Re: historicize Oracle RDBMS with Hadoop

Hi,

the my idea is modify the following scenario using Hadoop.

 

1) Actually every day I must historicize  the data from an SOURCE RDBS ORACLE to TARGET RDBS ORACLE with data pump

2) I have also many schedeluded BATCH that read data form SOURCE RDBS ORACLE  to store them in fileSystem : thi data are then

    moved on other machine with File Trasfer.

 

 

I would want look over the idea to use HDFS of hadoop for  import/export data from the SOURCE RDBS ORACLE for the point 1).

Read data with some tools of HADOOP  from  SOURCE RDBS ORACLE and import in HDFS for the point 2.

 

But i must understood  the right tools to use (SCOOB ?) and if this new approach is more performance than the current.

 

Thank in advance

Re: historicize Oracle RDBMS with Hadoop

Contributor

Sqoop can be used to move tables from Oracle to Hadoop.

Used with --direct option (or with OraOop driver: http://www.cloudera.com/content/cloudera/en/downloads/connectors/sqoop/oracle/v1-6.html ), it is quite performant.

 

I recommend giving it a try and seeing if it fits your requirements.

Highlighted

Re: historicize Oracle RDBMS with Hadoop

Hi,

 

but if wolud0t want to use more the RDBS TARGET ORACLE to hisotricize the data, is possible import the file after the MAP/REDUCE elaboration in HIVE ?

 

So i will use the RDBS SOURCE ORACLE but  the export data from HDFS in HIVE...!!!!!!

 

Another consideration: there are the tool in HADOOP framework  to use for File Transfer  ? 

 

Thanks in advance

 

Michele

 

 

Highlighted

Re: historicize Oracle RDBMS with Hadoop

Someone con you hepl me , please ?

 

Thank in advance

 

Michele

Highlighted

Re: historicize Oracle RDBMS with Hadoop

Master Guru
Using CDH (Sqoop, MR, etc.) would benefit you in your task iff you are seeing a lot of time spent in actually processing/consolidating your data before you ship/insert it back to the target RDBMS.

If the majority of your current time is spent instead on the insert/read speeds to/from the RDBMS, using Sqoop will not improve the situation as we'd be doing it in the same way.
Don't have an account?
Coming from Hortonworks? Activate your account here