Support Questions

Find answers, ask questions, and share your expertise

Need Help About Apache NiFi

avatar
New Contributor

Hello,

I have MsSql DB and Oracle DB. I am gonna try to explain what I want to do. I am fetching data from MsSQL DB than I am inserting that rows to Oracle DB but this structure not dynamic. Because I created TEST table on both side. So I am writing to table name of processor configuration. I want to create dynamic structure so when I start to flow, I need to migrate all database tables with all rows. But at the same time all tables need to create by processor automatically and when new rows came to MsSQL DB, the flow need to insert only new rows. Any one knows how to do that on Nifi 2.0.0-M3 ?

3 REPLIES 3

avatar
Community Manager

@omeraran Welcome to the Cloudera Community!

To help you get the best possible solution, I have tagged our NiFi experts @MattWho @mburgess  who may be able to assist you further.

Please keep us updated on your post, and we hope you find a satisfactory solution to your query.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community:

avatar
Master Mentor

@omeraran 

This use case sounds like a dataflow that would consist of the following:

GenerateTableFetch --> ExecuteSQL --> <any processor you may want to modify, extract, etc content if needed) --> PutDatabaseRecord

GenerateTableFetch will ingest rows from your source MySQL DB and maintain use NiFi state to record the maximum values for records so that it can continue to check for any ingest additional rows added.   It generates FlowFiles that contain the SQL queries needed by the ExecuteSQL to fetch the rows.   I don't know if your use case requires any manipulation, routing modifying, etc, but if so you would do that next.  And finally use the PutDatabaseRecord to write your rows to the Oracle DB. 

Please help our community thrive. If you found any of the suggestions/solutions provided helped you with solving your issue or answering your question, please take a moment to login and click "Accept as Solution" on one or more of them that helped.

Thank you,
Matt



avatar
Community Manager

@omeraran Has the reply helped resolve your issue? If so, please mark the appropriate reply as the solution, as it will make it easier for others to find the answer in the future. Thanks.


Regards,

Diana Torres,
Community Moderator


Was your question answered? Make sure to mark the answer as the accepted solution.
If you find a reply useful, say thanks by clicking on the thumbs up button.
Learn more about the Cloudera Community: