Support Questions
Find answers, ask questions, and share your expertise
Announcements
Alert: Welcome to the Unified Cloudera Community. Former HCC members be sure to read and learn how to activate your account here.

Real time data ingestion from Mssql

Highlighted

Real time data ingestion from Mssql

Explorer

What are the possible ways to ingest/replicate MsSql data into HDFS/Oracle. I know GG is one of the option but its no more supported by Oracle and Microsoft due to licensing conflict. Please suggest what others possible wasy we can do that.

Many Thanks

3 REPLIES 3
Highlighted

Re: Real time data ingestion from Mssql

Expert Contributor

You can use HDF to ingest MsSQL data to HDFS.

Highlighted

Re: Real time data ingestion from Mssql

Explorer

plz tell me in details?

Re: Real time data ingestion from Mssql

Contributor

@Bhupendra Mishra

HDF / Apache NiFi should do exactly what you need. With over 170 processors you will be able to connect do your database and move the data to Oracle/HDFS with drag & drop functionality.

https://nifi.apache.org/

http://hortonworks.com/products/data-center/hdf/

Your going to want to use QueryDatabaseTable processor, do your processing, and then use a PutHDFS processor. Below are some links, one with a full tutorial, the other with more information. You could also use ExecuteSQL to get data from your SQL database too.

This tutorial shows taking RDBMS data and putting to a file. The only difference for your use case is instead of the PutFile processor, just use a putHDFS processor. This should work for you:

http://www.batchiq.com/database-extract-with-nifi.html

Here is a similar question with a few answers & more info:

https://community.hortonworks.com/questions/36464/how-to-use-nifi-to-incrementally-ingest-data-from....

Don't have an account?
Coming from Hortonworks? Activate your account here